How A Google Change May Mistakenly Turn Search Traffic Into Referral Traffic

Google’s about to make a change to how it reports referrer information for those using its Chrome browser. As a result, some analytics programs may begin listing search visitors as if they instead came directly from Google without doing a search, though major packages will probably adjust OK.

The change was posted on the Google Webmaster Central blog yesterday, and it took some follow up to really understand what’s happening. Come along, and I’ll explain more.

Google & Blocking Referrers

Referrers are sort of a Caller ID for web browsers. They tell a web site where someone came from. For example, if you click on a link from one page to visit the next, the page you were on is passed along as referrer information that can be seen using web analytics tools. Sometimes this is also called “referer” information, due to a long-ago misspelling around the referrer standard. “Referral” is also sometimes used.

Last October, Google began blocking referrer information from being passed along by those searching on its search engine, if they were signed-in and using a secure connection.

Google said the change was made to better protect privacy. It turned out to be a precursor to preventing “eavesdropping” of especially private searches that might happen as part of Search Plus Your World.

However, despite saying the move was to protect privacy, Google went out of its way to continue passing along referrer data to paid advertisers. Other loopholes also remain. The move is incredibly hypocritical. See the articles at the end of this story to understand more about the blocking and the hypocrisy in greater depth

If Google is already withholding search term data for signed-in users, then what else could it really pull back? How about reporting even if a search happened.

Beginning in April, Google’s going to begin using the referrer meta tag to report what it calls a “simplified” referrer. The tag will let it override the real referrer that would go out, even what’s left of that referrer after search terms have been stripped.

How The Referrer Meta Tag Turns Searches Into Referrals

Consider a search for “hotels.” If you do that search and click on one of the top listings, say for Travelocity, the actual URL you’re going to looks like this:

https://www.google.com/url?sa=t&rct=j&q=hotels&source=web&cd=1
&ved=0CJABEBYwAA&url=http%3A%2F%2Fwww.travelocity.com%2FHotels
&ei=EftoT6eRLaKxiQK5uYGSBw&usg=AFQjCNHw3v58SOrf2HWCsE6AraxFouCmJQ

The URL doesn’t lead directly to the site. Instead, it redirects through Google itself, in a way that Google can record what’s in the URL to better track the click.

I’ve bolded how Google embeds in the URL information that someone searched for the word “hotels” and clicked on the first listing in the results, which in turn took them to the page at Travelocity, also shown in bold.

If this search is done when someone is signed-in using a secure connection, Google drops the search term portion. It basically looks like this:

https://www.google.com/url?q=&url=http%3A%2F%2Fwww.travelocity.com%2FHotels

An analytics program can tell that a search happened by seeing the “q=” part in the URL, but the actual term itself has been stripped out by Google. So while Google Analytics can’t report what the search words were (and thus says “not provided”), it still can tell that a search happened.

The new change takes out everything but the start of the referrer. Do a search on Google.com using Chrome, and this is all that will be reported:

https://google.com

Because there’s no indicator that a search happened, an analytics program may interpret that people have come from a link on Google.com rather than doing a search there. This means that search traffic would mistakenly get recorded as what’s called “referral” traffic.

Search Traffic Vs. Referral Traffic

To understand better, here’s my traffic breakdown to my personal blog Daggle from yesterday. This is from Google Analytics:

You see that 76% of my traffic was from search, people who did some type of recognized search and visited my site. Google Analytics doesn’t know the actual search terms for nearly a third of these visits (yeah, wow, right?). See how “not provided” makes up 35% of all keywords in the lower chart? But Google Analytics still knows that they were searches, so they get counted into the overall search total.

After that, about 14% of traffic is from referrals, people who clicked on a link from one site that lead to my own. Another 10% of traffic is direct, people who either directly entered the URL of one of my pages into their web browser or who came to my site without any referrer information being reported (which isn’t necessarily direct traffic, but it gets counted that way).

With the change, Google Analytics or other analytics program would count some of my search visits as if they are referral visits, unless they adjusts for this. The slice of search traffic would start to drop, even though my search traffic could potentially be going up.

Google Analytics Will Adjust, Other Vendors Being Told

If you use Google Analytics, Google says there’s no reason to panic. Google Analytics is supposed to figure out how to count things correctly. The same may be true for other vendors, by the time this happens. Google told us:

We’re using the meta referrer standard which allows us to choose the origin and still send a referrer to http sites from https search results (without going through a redirect on an http host).

Google Analytics will also adjust for this change, and we’re in the process of reaching out to a number of other analytics vendors to notify them about this in advance.

Only Impacts Chrome & Really A Time Saver?

The change will only happen for those using Google Chrome, as that’s the only browser that supports the meta referrer tag, Google told us. As for why bother doing this at all, the blog post says:

This results in a faster time to result and more streamlined experience for the user.

I’m a bit doubtful about the savings. It’s not like Google is stopping the actual click tracking that it does. Everything you click on still gets redirected, which causes a tiny delay. The meta referrer tag only means that those using the Chrome browser will pass along a shorter URL for where they came from.

Surely that’s not saving much time? I asked Google how much this really speeds things up:

We don’t have data to share right now. However, this does allow the user’s browser to avoid making an extra connection to http://www.google.com (which the browser may not have already established since the search was on https://www.google.com/).

I’m still confused about why the browser would make an extra connection back to Google after someone has left, because of anything to do with passing along referrer data. I’ll check on that.

Pleading Again For More Data In Google Webmaster Central

Overall, there’s probably no reason to panic, if you use a major analytics provider. But it’s something you should check on. It’s also an unpleasant reminder that Google keeps messing with the referrer data that it provides to publishers in a way that messes up their trending.

Google’s answer to all these changes is that people should make use of Google Webmaster Central to pull in missing search data. But that data only goes back 30 days. That does nothing to restore the trends that have been destroyed since withholding began.

I’ll repeat what I said earlier this year about all this:

I think Google should do more than 60 days. I think it should be providing continuous reporting and holding that data historically on behalf of sites, if it’s going to block referrers. Google is already destroying historical benchmarks that publishers have maintained. Google’s already allowed data to be lost for those publishers, because they didn’t begin to go in each day and download the latest information.

So far, all Google’s done is provide an Python script to make downloading easier. That’s not enough. Google should provide historical data, covering a big chunk of the terms that a site receives. It’s the right thing to do, and it should have been done already.

See the articles below for further background about the blocking:

• Google To Begin Encrypting Searches & Outbound Clicks By Default With SSL Search
• Google Puts A Price On Privacy
• Google’s Results Get More Personal With “Search Plus Your World”
• 2011: The Year Google & Bing Took Away From SEOs & Publishers