RIP Referrer Data! How Mobile Apps Can Kill Your Mobile Metrics

Imagine justifying your PPC budget, if you could no longer track which keywords sent each visitor. Regardless of how successful your program is today, you probably would not think twice about killing or dramatically shrinking it tomorrow. And for pretty good reason, right? That referral data is connective tissue that enables you to attribute success and allocate resource against ROI metrics.

Well, in the nascent mobile ecosystem, increasing amounts of this referrer information are becoming invisible to your mobile clickstream and ROI analysis. Maybe you have noticed? If not, you probably will soon, as apps from Facebook, Google Maps, Google Search, Twitter, Yelp, Groupon, Foursquare, QR scanners, and many more, continue to explode in popularity and usage.

The issue at hand is pretty simple. Smartphone apps are innately unable to pass “referring page” information to your site analytics, for the same reason Word docs or PDFs are not recorded as website referrers: they’re separate applications. Even for mobile apps that embed browsers (like Facebook, Google, Twitter, Groupon), there simply is no “referring page” data to pass upon click-through.

If you’re expecting to see this traffic nicely categorized in some analytic report, you’ll be waiting a while. (Of course, tracking tags can be appended to URLs for ad campaign landing page referrers, but that’s not easily achieved with mobile channel “earned” media.)

The net result is that your “direct” traffic and sales counts are inflated, and that a sizable percentage of your actual mobile search and mobile social traffic is understated. That means you are likely under-investing in mobile, and giving conversion credit to other channels to boot. That, my friend, is no way to grow mobile business! (And I suspect this is a contributing factor in why 79% of advertisers have not invested in mobile optimized site.)

For brands with strong search and social presence, Facebook, Google, and Twitter apps are the most commercially-relevant apps likely to be sending volumes of misclassified visitors your way. Here’s a look at how “referrers” from these apps get recorded in both Google Analytics and standard log files.

Facebook App

As the most popular app for iPhone, Blackberry, and second most popular on Android, nearly 50% of Facebook’s 500 million worldwide members check and update status from their devices. What gets recorded when these users click your company URL, status posts, or a deep link that was shared by a friend on Facebook?

Starbucks' Facebook App Profile Page

Starbucks' Facebook App Profile Page

Starbucks' Facebook App Comments Page

Starbucks' App Comment Page

 

The first click from within the Facebook app to a web URL initiates a new browser session. The request gets recorded by Google Analytics as a “direct” hit to the “/” home page (regardless of actual page requested), and without referrer. Bummer.

Interestingly, upon the second click within the app browser, the “referrer” gets populated with “Facebook.com” through the ever-so-helpful “l.php” redirect script page path. Unfortunately, this is exactly how desktop Facebook clicks get recorded too, so you cannot easily tell this traffic came from the Facebook app.

Digging into log files provides slightly more illumination – though not much. Facebook is a fairly stingy masker of referral data. The first page requested gets logged without a referring page, for reason explained above. (Although, Facebook could probably populate this field with “Facebook App” or something helpful, without passing member URLs or private information.)

The original request is also logged with the smartphone’s native browser user agent string:

Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Mobile/8J2″)

However subsequent in-app browser page requests replace the browser string with:

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

So to get a true reading on mobile Facebook app traffic, log file data-mining is probably in order. This can get a bit complicated, as you can pivot around the user agent data, yet you need to filter out the 301 and 206 status codes that were requested from the same user agent string. (These requests are what populate the Facebook page “snippit” and shouldn’t be counted as visitors.)

Google Search App

Maps is the second most popular smartphone app. Not far behind is the Google Search app. Mobile users are search dominant, and mobile query volume has continued to grow inline with smartphone growth. We should see mobile drive 22% of total query volume by year-end. How is this search app traffic recorded on your site? In the same way that Maps users are.

 

Google Maps Info Page

Google Maps Info Page

Maps users who click the “home page” link from your Maps “Info” launch a new browser session. Hence they are recorded as “direct” traffic. So are users of the Google search app. If you’re running a mobile PPC campaign, you presumably have tracking parameters appended to your landing page URLs that will tell your analytics this was a mobile search query (not to mention data available from Google’s campaign reports).

But for mobile organic, you’re out of luck. Not only does Google Analytics not record Google as the referrer, the mobile keywords that drove traffic also go incognito. You’ll have to mine your log files to count them.

Google App Search Results

Google App Search Results

Fortunately, Google is more forthcoming than Facebook with passing page request data. Google populates the referring page field with a few new parameters, identifying the query being made from “mobilesearchapp” for instance:

http://www.google.com/m/search?uaction=k&safe=images&client=mobilesearchapp&q=volleyball%20shoes%20mens&hl=en&v=0.8.2.6553&ie=UTF-8&source=mobilesearchapp&channel=iss&vse=1&rlz=1MDAPLA_en-USUS423

Unlike with the Facebook app, where only the first page request is logged with the smartphone’s proper user agent string, all page requests from within the Google app browser appear to properly populate the user agent string. That makes it easier to mine your log files and true up the numbers.

Twitter App

About 50% of Twitter’s traffic is mobile. To be mobile is to be social. Twitter app traffic faces the same analytic peril that Facebook and Google do, but with a double-whammy twist. Not only is there no page referrer data sent from app to browser, but most all Twitter referral traffic will be routed through URL shorteners that 301 redirect, killing page referrer data dead in its tracks for Google Analytics.

 

Tweet Consumed via Twitter App

Tweet Consumed via Twitter App

 

Target Site via Tweet Within Twitter App Browser

Target Site via Tweet Within Twitter App Browser

Of course, shorteners like bitly will record Twitter as traffic source (or Facebook, etc). But it’s worth noting, these URL analytics also record app-referred traffic as “direct” traffic. So don’t be mistaken – when you see mobile.twitter.com or m.facebook.com referred traffic in your bit.ly stats, that’s mobile web traffic they’re talking about, not mobile app traffic.

What’s in the log files? Twitter does something interesting here with the requesting browser field. They modify the user agent string by adding (at least for iOS) “Twitter for iPhone” at the end, eg:

“Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Mobile/8J2 Twitter for iPhone

This is pretty helpful for data mining. It would be nice for this approach to become standard practice for app developers: It sends both the base browser data, as well as the app which “referred” the session. Analytic providers may then be able to report each app browser’s requested landing pages more precisely.

Audit Your Analytics

The bottom line is this: if 10-20% of your Facebook, Google Search, and Twitter visitors are coming from their respective apps, that starts adding up to significant business that should count towards your mobile ROI. Audit your analytics (and log files) to adjust your reported metrics accordingly.

While you’re at it, pay attention to how customers are using emerging apps like Yelp, Groupon, Foursquare, or QR Scanners to access your site. Each have their own idiosyncrasies for identifying visitors referred.

You can’t manage what you can’t measure. Until you can measure the impact mobile apps are having on your mobile ROI, it’s not difficult to imagine you’re leaving money on the table.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: Mobile | Mobile Search

Sponsored


About The Author: is founder and president of Pure Oxygen Labs, a consulting and mobile technology firm. Brian's been in search marketing for over 10 years, most recently leading SEO firm Netconcepts through to acquisition by Covario. Follow Brian on Twitter @brianklais.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://corvalence.blogspot.com Joseph Urban

    Good article. This is a decent explanation for our growing Unknown category in our referrers reporting.
    One missing point on twitter traffic: Twitter apps hit the URL shorteners regardless if the user clicks through. I believe they are automatically assessing the legitimacy of the link. So, Bit.ly and others will reflect a set of highly inflated traffic metrics. I’ve seen 9X multiples on some weeks. I’ve been able to verify this because we do append our tweets with a parameter that we can track.

 

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide