How A Google Change May Mistakenly Turn Search Traffic Into Referral Traffic

google analytics iconGoogle’s about to make a change to how it reports referrer information for those using its Chrome browser. As a result, some analytics programs may begin listing search visitors as if they instead came directly from Google without doing a search, though major packages will probably adjust OK.

The change was posted on the Google Webmaster Central blog yesterday, and it took some follow up to really understand what’s happening. Come along, and I’ll explain more.

Google & Blocking Referrers

Referrers are sort of a Caller ID for web browsers. They tell a web site where someone came from. For example, if you click on a link from one page to visit the next, the page you were on is passed along as referrer information that can be seen using web analytics tools. Sometimes this is also called “referer” information, due to a long-ago misspelling around the referrer standard. “Referral” is also sometimes used.

Last October, Google began blocking referrer information from being passed along by those searching on its search engine, if they were signed-in and using a secure connection.

Google said the change was made to better protect privacy. It turned out to be a precursor to preventing “eavesdropping” of especially private searches that might happen as part of Search Plus Your World.

However, despite saying the move was to protect privacy, Google went out of its way to continue passing along referrer data to paid advertisers. Other loopholes also remain. The move is incredibly hypocritical. See the articles at the end of this story to understand more about the blocking and the hypocrisy in greater depth

If Google is already withholding search term data for signed-in users, then what else could it really pull back? How about reporting even if a search happened.

Beginning in April, Google’s going to begin using the referrer meta tag to report what it calls a “simplified” referrer. The tag will let it override the real referrer that would go out, even what’s left of that referrer after search terms have been stripped.

How The Referrer Meta Tag Turns Searches Into Referrals

Consider a search for “hotels.” If you do that search and click on one of the top listings, say for Travelocity, the actual URL you’re going to looks like this:

http://www.google.com/url?sa=t&rct=j&q=hotels&source=web&cd=1 &ved=0CJABEBYwAA&url=http%3A%2F%2Fwww.travelocity.com%2FHotels &ei=EftoT6eRLaKxiQK5uYGSBw&usg=AFQjCNHw3v58SOrf2HWCsE6AraxFouCmJQ

The URL doesn’t lead directly to the site. Instead, it redirects through Google itself, in a way that Google can record what’s in the URL to better track the click.

I’ve bolded how Google embeds in the URL information that someone searched for the word “hotels” and clicked on the first listing in the results, which in turn took them to the page at Travelocity, also shown in bold.

If this search is done when someone is signed-in using a secure connection, Google drops the search term portion. It basically looks like this:

http://www.google.com/url?q=&url=http%3A%2F%2Fwww.travelocity.com%2FHotels

An analytics program can tell that a search happened by seeing the “q=” part in the URL, but the actual term itself has been stripped out by Google. So while Google Analytics can’t report what the search words were (and thus says “not provided”), it still can tell that a search happened.

The new change takes out everything but the start of the referrer. Do a search on Google.com using Chrome, and this is all that will be reported:

https://google.com

Because there’s no indicator that a search happened, an analytics program may interpret that people have come from a link on Google.com rather than doing a search there. This means that search traffic would mistakenly get recorded as what’s called “referral” traffic.

Search Traffic Vs. Referral Traffic

To understand better, here’s my traffic breakdown to my personal blog Daggle from yesterday. This is from Google Analytics:

You see that 76% of my traffic was from search, people who did some type of recognized search and visited my site. Google Analytics doesn’t know the actual search terms for nearly a third of these visits (yeah, wow, right?). See how “not provided” makes up 35% of all keywords in the lower chart? But Google Analytics still knows that they were searches, so they get counted into the overall search total.

After that, about 14% of traffic is from referrals, people who clicked on a link from one site that lead to my own. Another 10% of traffic is direct, people who either directly entered the URL of one of my pages into their web browser or who came to my site without any referrer information being reported (which isn’t necessarily direct traffic, but it gets counted that way).

With the change, Google Analytics or other analytics program would count some of my search visits as if they are referral visits, unless they adjusts for this. The slice of search traffic would start to drop, even though my search traffic could potentially be going up.

Google Analytics Will Adjust, Other Vendors Being Told

If you use Google Analytics, Google says there’s no reason to panic. Google Analytics is supposed to figure out how to count things correctly. The same may be true for other vendors, by the time this happens. Google told us:

We’re using the meta referrer standard which allows us to choose the origin and still send a referrer to http sites from https search results (without going through a redirect on an http host).

Google Analytics will also adjust for this change, and we’re in the process of reaching out to a number of other analytics vendors to notify them about this in advance.

Only Impacts Chrome & Really A Time Saver?

The change will only happen for those using Google Chrome, as that’s the only browser that supports the meta referrer tag, Google told us. As for why bother doing this at all, the blog post says:

This results in a faster time to result and more streamlined experience for the user.

I’m a bit doubtful about the savings. It’s not like Google is stopping the actual click tracking that it does. Everything you click on still gets redirected, which causes a tiny delay. The meta referrer tag only means that those using the Chrome browser will pass along a shorter URL for where they came from.

Surely that’s not saving much time? I asked Google how much this really speeds things up:

We don’t have data to share right now. However, this does allow the user’s browser to avoid making an extra connection to http://www.google.com (which the browser may not have already established since the search was on https://www.google.com/).

I’m still confused about why the browser would make an extra connection back to Google after someone has left, because of anything to do with passing along referrer data. I’ll check on that.

Pleading Again For More Data In Google Webmaster Central

Overall, there’s probably no reason to panic, if you use a major analytics provider. But it’s something you should check on. It’s also an unpleasant reminder that Google keeps messing with the referrer data that it provides to publishers in a way that messes up their trending.

Google’s answer to all these changes is that people should make use of Google Webmaster Central to pull in missing search data. But that data only goes back 30 days. That does nothing to restore the trends that have been destroyed since withholding began.

I’ll repeat what I said earlier this year about all this:

I think Google should do more than 60 days. I think it should be providing continuous reporting and holding that data historically on behalf of sites, if it’s going to block referrers. Google is already destroying historical benchmarks that publishers have maintained. Google’s already allowed data to be lost for those publishers, because they didn’t begin to go in each day and download the latest information.

So far, all Google’s done is provide an Python script to make downloading easier. That’s not enough. Google should provide historical data, covering a big chunk of the terms that a site receives. It’s the right thing to do, and it should have been done already.

See the articles below for further background about the blocking:

Related Topics: Channel: Analytics | Features: Analysis | Google: Analytics | Top News

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://twitter.com/zach1987johnson Zach Johnson

    Go Organic

  • http://www.livinginthailand.net/ Neale

    This has been along time in the coming, unless you are super savvy it is going to get harder and harder for the average joe to glean info on how visitors arrive at your website.

  • ewanseo

    Thanks for sharing this info Danny. It is a concern as the less info we see about where searches are coming from the harder our job as SEO internet marketers becomes. Let’s hope, as you say. ‘
    Overall, there’s probably no reason to panic’

  • http://usman.co/ Usman Latif

    WTH! Another step towards monopolizing the access to referral data.

  • Matt Menke

    I’m no expert, but here’s my (possibly incorrect) understanding:

    The HTTP connection is needed because when navigating cross domain from an HTTPS source page, no referrer is sent for security reasons, since it’s not uncommon to have private information as part of the URL.  As a result, if Google want to send any referrer tag, it currently has to go through an HTTP page.  As Google itself uses HTTPS, that means you need to redirect from the search page through an HTTP link.  With the ability to set the referrer from their HTTPS search page, the HTTP connection should no longer be needed.  Presumably Google uses the “ping” attribute of the “a” tag to do its own bookkeeping of what was clicked, or something similar.

  • http://www.ramkrshukla.com/ Ram Krishna Shukla

    Great stuff as usual good observation, I love the article of Avinash Kaushik to visualize such data http://www.kaushik.net/avinash/google-secure-search-keyword-data-analysis/

  • http://www.webstatsart.com/ Webstats Art

    I wish twitter would do something because it is awesome and needs to allow us to upload web content. I would spend tons of time on twitter if I could put more content there. http://www.webstatsart.com/twitter

  • http://www.realtimetricks.com/ RealTimeTricks

    This is truly awesome if only if it gets successful. :P

  • http://twitter.com/NickStamoulis Nick Stamoulis

    It’s always frustrating when data is moved around. It makes it that much harder to do our jobs as SEO professionals well because we are suddenly working in the dark. Thanks for the heads up. 

  • http://twitter.com/daliburgado Dali Burgado

    Thank you for always sharing such timely info, Danny! And just when you thought (not provided) was getting worse… Come on Google!  

  • http://www.facebook.com/terry.stanfield3 Terry Stanfield

    Thanks for the article. As an SEO guy, it is challenging to provide information on what organic terms are converting to leads if a third of the conversions are (not provided) or (not set). I can make educated guesses but it is nice to empirical data!

    Even using the webmaster tools it just shows what keywords were used to get to my site but not which terms converted.

  • Jeremy Muratore

    Thanks for calling this out in great detail. Stripping out the referrer seems to be passive-aggressively “forcing” adoption of Google Webmaster Central and paid advertising for search initiatives. They have us all following the cheese through the Google ecosystem maze and blurring SEO/SEM/Social lines. Will Google just start implementing automated tools & services that act on information privy only to them?

  • http://rajeevkumarsinghblog.blogspot.com/ Rajeev Kumar Singh

    Google’s monopoly over the search making them devil! 

    How much this speed up me – not much. Why Google need to force these things? Where “don’t be evil” motto goes?

  • http://twitter.com/incrediblehelp Jaan Kanellis

    “Google’s answer to all these changes is that people should make use of Google Webmaster Central to pull in missing search data. But that data only goes back 30 days. That does nothing to restore the trends that have been destroyed since withholding began.”

    Danny if you have synced your GWT account to GA  and then access the data your looking for in GA NOT GWT it goes back more than 30 days.

  • http://www.lattimore.id.au/ Alistair Lattimore

    Danny,

    The HTTP REFERER header can only get set when using HTTPS if the URL you’re navigating to is on the same origin/host. 

    For example, if you navigated from a HTTPS URL on Search Engine Land to a HTTP URL on Search Engine Land, the HTTP REFERER header would get set. If you navigated from a HTTPS URL on Search Engine Land to a HTTP URL on a different host, say http://www.google.com – no HTTP REFERER header will get set at all.

    The redirect that Google have at the moment for logged in users provides a mechanism for them to provide some tracking (a blank q query string parameter for instance) instead of nothing at all, which is what would happen by default if you searched while logged in and they didn’t redirect through a HTTP redirect on the same host.

    John Mueller did reply to my question on the official Google announcement, however only answered part of my question. The other ‘benefits’ outlined in the announcement, such as automatically simplifying the referring URL and a streamlined experience for the user seem nothing more than smoke to me in practical terms.

    I personally think this is another change, a wolf in sheep’s clothing if you will, for something larger happening in the future. I appreciate the sentiment of improving performance for users by not requiring the intermediate redirect but you’re talking about a few milliseconds as it redirects through Google-speed infrastructure. 

    Al.

  • miguelgarza

    I found this article when trying to figure out if my Chrome install had some sort of infection. Because of the redirection URL. Just installed Firefox because Chrome is also taking a really, really long time to load pages. Firefox much better by nearly ten seconds. I wanted Chrome to be my go-to browser but I’m afraid Google’s desire for innovation is fighting with its desire for profit, and it shows.

  • http://www.facebook.com/debbie.c.webb Deb Webb

    I’m with Alistair on this one. Maybe Google is trying to divert our attention with a bogus tweak while they deploy their next move.
    I really enjoyed reading this article.  I was wondering what the meant. You wrote this in such an easy to understand way.
    Diversion or just a silly move by Google?

  • Vince Heilman

    “.. a new proposal to reduce latency when a user of Google’s SSL-search clicks on a search result with a modern browser such as Chrome. ” 

    Is this just for the SSL or https:// google account, logged in users?  The ones that are currently “Not set”? 

  • http://pulse.yahoo.com/_CM4IAXUNJMDYHMVQPJZXQNEHPM ben

    Sounds like future analytics tools will have to pull data from webmaster tools and any other available sources and then compile it together, rather than relying solely on weblogs or tracking pixels.

  • http://twitter.com/benboombastic Benedict Hayes

    Hey Danny,

    Do you buy this speed c%#p?

    Is it me or is this not just the machine doing straight forward monopoly play.

    All third party analytics packages are rendered useless at keyword tracking (well at least inferior to Google analytics). Only Google tools will give you data, but WMT has sampled data so is inaccurate in any case.

    This obviously sods up every SEO as how the hell do you know if you are getting traffic or not from your efforts.

    Its funny how only people who pay Google exorbitant amounts of money (on a system designed to fleece everyone in the auction because small business owners don’t understand it and jack up the bids prices) will be able to see which keywords work.

    Lets face it before long most people on the planet will somehow be logged into Google, even if they don’t realise it – so it looks as if keywords are to be given a grave stone and become a long forgotten history of the net.

    How or why are Google allowed to make changes like this that affect businesses the world over.

    Is it an effort to stop SEO or is it an effort to affect competitors in analytics or both?

    or am I being massively paranoid…

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide