Safari Shifts To Google Secure Search in iOS 6, Causing Search Referrer Data To Disappear

google-g-logo-2012Another source of Google search referrer data appears lost. Searches through Google that happen in Apple’s Safari browser search box in iOS 6, Apple’s latest mobile operating system, no longer pass along search terms to publishers.

For those unfamiliar, “referrer” data is information a browser passes along to a publisher about the last page you were on. When you do a search on a search engine, this means what you searched for — the exact words — are passed along as part of the referrer.

The change, as we’ve learned as this story has developed, is because Apple apparently — as part of iOS 6 — now routes searches to Google made through the Safari search box to an encrypted version of Google search. Google told us:

The web browser on iOS 6 switched to use SSL by default and our web servers don’t yet take that fact into account. Searching still works fine, but in some situations the HTTP referer header isn’t passed on to the destination page. We’re investigating different options to address this issue.

Better Privacy, But Ad Clicks Still Open To Eavesdropping

Encryption adds some privacy protection for searchers, but it also means publishers no longer understand how these searchers found their web sites through Google, unless they are Google advertisers. That loophole also means that some searches are still vulnerable to potential “eavesdropping” by others.

Google offers two types of encrypted search. Google Encrypted Search blocks referrers from being sent — and thus removes any eavesdropping worries — for all clicks that leave the Google web site, unless those clicks lead to another encrypted site (which means the destination site itself can see the search terms but no one else can eavesdrop on the connection).

Apple is making use of the Google SSL Search, which Google introduced last year. This blocks eavesdropping when people click on unpaid listings. However, any clicks on paid listings are transmitted in the clear, leaving that data open to eavesdropping.

Firefox made a similar move to Google SSL Search earlier this year, and Google itself shifted all searches for those signed-in to Google to use Google SSL Search a year ago (at least for desktop browsers — on mobile browsers, SSL doesn’t appear to be the default after signing in).

I’ve had a fairly big issue with Google for compromising privacy with Google SSL Search by not blocking all referrers as it could, for what appears to be the sake of not upsetting advertisers. For more about that, see these two articles:

When Google Does & Doesn’t Block

Ryan Jones noticed earlier this week that searches done on Google, through Safari’s search box on iOS 6, no longer passed along referrer data. We tested ourselves today and found this also to be the case.

This only happens when you use the search box in Safari, not if you go to Google directly. That’s important to understand, because if you don’t test the right way, you’ll still see referrers.

Here’s an easy way anyone can test, drawn from some of the comments below.

If you search from the Google home page in Safari for whatismyrefer, then click on the first result, that will bring you to to the What Is My Referer Site, where your referrer data is shown:

If you do the same thing using the built-in search box in Safari, if you search against Google (which is the default in the US), no referrer is passed:

FYI, “referer” is a long ago misspelling for “referrer” in HTML specs, which is why you sometimes still see that misspelling used.

Bing & Yahoo Referrers Not Blocked

Doing the same test above for Bing and Yahoo finds referrers are passed either if you go to their home pages or if you change your settings to make them the default for Safari’s search box. That’s probably because neither has an encrypted search service that Apple could use, which would have blocked referrers.

Another Referrer Source Gone & A Mystery

Soon after Google shifted to Google SSL Search for signed-in users last year, the move quickly caused some publishers to see more than 20% of their search referrer data to disappear. Those who use Google Analytics, for example, are now familiar with how one of their top search terms is no “not provided.”

Currently, we find just over half our searches from Google to Search Engine Land now are “not provided” or have search term referrer data removed.

That leads to a mystery we’re still exploring. Google SSL Search should still pass a referrer, just one that has the search terms removed. That means a tool like Google Analytics can tell that a search was done on Google — and count this traffic as search traffic — but just not know what the exact term is (hence the “not provided” message).

In the case of Safari, no referrer at all is passed. Analytics programs can’t tell at all that the traffic came from Google, so it’s counted as “direct.”

If Apple were sending searches through Google Encrypted Search, then it would make sense for referrers to be completely stripped. However, that’s not the case. I can tell this because the Google URL shown on search pages begins https://…. (which is Google SSL Search) rather than https://encrypted.google.com…. (which is Google Encrypted Search).

I think the answer comes from something Ryan Jones notes below. In Google SSL Search on a mobile browser, Google doesn’t appear to be redirecting clicks. Since it doesn’t do that, it’s not able to override how a browser wants to remove a referrer according to the https standard.

Normally, if you go from a secure server to a non-secure one, no referrer at all is passed. Google SSL Search changes this behavior. If you click on a link, the link is actually rerouted back through Google, where it decides to do one of two things:

  • If it’s a click on an unpaid listing, Google removes the search term and passes along the referrer through an unsecure connection to the destination web site
  • If it’s a click on a paid listing, Google leaves the referrer alone and passes along the referrer through an unsecure connection

Since the mobile Google SSL Search apparently lacks redirection, the “normal” behavior takes over. Any click leaving Google to an unsecure web site gets no referrer. This means mobile Google SSL Search is actually more secure than desktop search. But few have been likely to use it until now, when Apple began pointing to it by default.

Now that this is happening, chances are Google will actually make mobile Google SSL Search less secure by restoring referrers to advertisers. Non-advertisers will likely get referrers again, too, but without search terms attached to them.

Postscript: We’ve updated this story to better clarify how referrers are passed if you search from the Google home page, as opposed to from the Safari search box. We’ve also updated it to reflect the confirmation from Google that we were given. Search Engine Land news editor Barry Schwartz was the original author of this story, and it was shifted to Danny Sullivan as he continued writing on it. It was also further updated to better explain why no referrers at all are showing in mobile Google SSL Search.

Postscript 2: See our follow-up story How An iOS 6 Change Makes It Seem Like Google Traffic From Safari Has Disappeared.

Related Stories

Related Topics: Apple | Channel: Analytics | Features: Analysis | Google: Analytics | Top News

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://twitter.com/roquelagellera Roque Lage de Llera

    Hi Barry, ¿How would this affect to analytics results?

  • Travis Prebble

    The effect would be that you’ll see a reduction in measured referrals from Google search, especially if you have a mobile heavy site. The traffic will still be there, will still come from Google, you just won’t know the source.

  • http://twitter.com/theseanstewart Sean Stewart

    It would show an increase in (direct) visits, and a decrease in organic visits from Google. The net change is 0.

  • http://andrewmichaelsmith.com/ Andrew Smith

    Are you’re sure that you’re not using google https on the iPhone 6? That would cause the same results.

  • http://twitter.com/numbakrrunch Charlie Reverte

    An easy way to reproduce this yourself is to search for “what is my referer” on iOS6 and click on the search result for whatismyreferer.com. You’ll see the referer is blank when you come from google and intact for bing and yahoo

  • http://twitter.com/TheTopTom Thomas Howard

    On my desktop (in Chrome), I switched my User-Agent to iOS 6, and typed “what is my referrer” into google. I then went to whatismyreferer.com and it was empty.

    The same steps under my regular Chrome User-Agent produced the referrer data.

    Both were completed while using the SSL version of Google.

    Therefore Google is the one causing this issue, not iOS 6.

  • http://twitter.com/BigBryC Bryan Cristina

    A net 0 for visits, a negative 100% for actually knowing what happened

  • http://twitter.com/roquelagellera Roque Lage de Llera

    Thats how i understood that….Pretty big issue dont you think??

  • http://twitter.com/TommyGriffith Tommy Griffith

    Well, the ‘change’ is less actionable data in webmasters’ analytics. I wonder how much of an impact this will be relative to [not provided] secure search data, which was about 20% of queries.

  • http://www.rustybrick.com/barry Barry Schwartz

    No, that would just block the query, not the whole referrer. See iOS5 vs iOS6 example above.

  • http://twitter.com/numbakrrunch Charlie Reverte

    It turns out that I was logged in to google for my the previous test. It looks like this is normal referer dropping from https seach, not something specific to iOS6. I keep forgetting about that :) Logging out and doing an http google search using the method above restores the referer. It looks like this article is a false alarm..

  • RyanMJones

    Thanks for sharing this Barry. I think the implications here are HUGE for analytics – especially if the trend catches on. I’ve popped into the FireFox forums from time to time and people keep bringing up the suggestion to completely strip referer information from FF as well. Just as we’re steering the industry away from rankings and toward KPIs, we may be forced to reverse path.

  • http://andrewmichaelsmith.com/ Andrew Smith

    As I understand it most browser implementations will block the referrer if it’s coming from a HTTPS site. Are you saying that they would just strip out the “q=” parameter? Because I don’t think that’s the case.

  • http://twitter.com/samuelcrocker Samuel Crocker

    This seems to be the case – if you try the same test but using the https:// version on iOS 5 (which is the default on the toolbar on iOS 6) you get the same result – no referer passed. This is different to the result you see on desktop searching on https:// however (because they seem to put you through a redirect and strip out query whilst still tracking you), so the issue is not with the OS but rather the change in the default toolbar behaviour from one OS to the next.

    It doesn’t change the fact that this change in the default will have a massive impact, but the issue is not really with the iOS per se.

  • http://andrewmichaelsmith.com/ Andrew Smith

    For people who worried about losing out on search terms there’s always http://www.google.com/webmasters/

  • http://twitter.com/samuelcrocker Samuel Crocker

    For what it is worth – based on our testing within our team just now this has more to do with mobile and https:// – the same result happens on android if you search on the SSL version of Google.

    As above, it seems like the issue is with handling of SSL on mobile and the fact that Google does not do the redirect with the query string stripped out rather than anything specifically to do with iOS 6. What iOS 6 has done, however, is highlight this point by changing the default behaviour within the toolbar to be through the https:// version.

  • http://www.rustybrick.com/barry Barry Schwartz

    Google SSL shows referred from search. It just doesn’t show the query (i.e. it shows not provided).
    Here, Google is stripping everything, as far as I understand.

  • http://www.facebook.com/profile.php?id=535260227 Jeff Kean

    Very big, same when google started striping out for signed in users. my analytics shows tons of (not set) or (direct) when there not. Its horrible

  • http://andrewmichaelsmith.com/ Andrew Smith

    What’s happening is that because the link is a HTTPS site the browser is not sending any referrer header. This happens on every HTTPS site and is part of the RFC. See the second paragraph under “Refer Hiding”: http://en.wikipedia.org/wiki/HTTP_referer

  • http://www.rustybrick.com/barry Barry Schwartz

    Meanwhile, I am now seeing referer data if you are signed in or out. Maybe Google changed it?

  • http://www.blastam.com Joe Christopher

    I agree with Samuel’s tests. On Google mobile results, the non-ssl redirect appears to not be present like it is from the desktop version. The non-ssl redirect on desktop searches (when searching from ssl) passes in referrer data and drops the keyword parameter. On mobile though, when searching via ssl, no non-ssl redirect exists.

    Much like how Firefox and more browsers are defaulting to https pages when available, this is more problematic for mobile in this case because of the lack of a non-ssl redirect.

  • http://twitter.com/BrianKlais Brian Klais

    This is big news for all marketers. Great find @rustybrick:disqus and @RyanMJones:disqus.

    I wrote a piece last year at SEL on how Google hadn’t yet secured mobile search. I speculated then SEOs had a brief window – maybe a year – to analyze mobile search intent and referrer data before it vanished. (http://searchengineland.com/give-thanks-google-hasnt-secured-mobile-search-data-yet-101819).
    So it isn’t surprising that this data is disappearing. What is surprising is if it’s Apple causing it. It’s possible that what we’re seeing is Google beginning to secure mobile queries. But if so, why just for iOS and not Android queries? Interested to see how this unfolds… but the bottom line is the same: the mobile search data window is now closing with each new install of iOS6.

  • http://andrewmichaelsmith.com/ Andrew Smith

    Google are powerful but I don’t think they can change the HTTPS spec on a whim :)

    I think what’s happening there is what @twitter-28535482:disqus is describing: In some cases (seemingly – when you are signed out) Google first point to a HTTP Google URL which then redirects you to the URL you’re after. Though I haven’t actually looked at what they do.

  • RyanMJones

    If only it would let us tie them to KPIs though. It’s not enough for me to see “how many” if I can’t see what they’re doing too.

  • t0rben

    Guess there are only webmasters here complaining about this? From a users perspective I think that’s a good thing. It’s none of your business what I googled before visiting your website imo.

  • RyanMJones

    On Wired, Google is doing a redirect though and passing the referer. It appears they’re NOT doing that on mobile.

  • RyanMJones

    I’m less interested in “what” you googled and more interested in the fact that you came from Google. That’s what we’re losing now for mobile searchers.

  • http://crem.in Jonathan Cremin

    You can’t pass the referrer from HTTPS to HTTP, there’s nothing you can do to change that. This is not a decision that Google made, it’s a side effect of the technology.

  • RyanMJones

    Correct, however on the web when you search on https, google redirects to an intermediate http page which then passes a referer. Mobile doesn’t seem to be doing that.

  • http://crem.in Jonathan Cremin

    No it doesn’t, the intermediate page is also served over SSL. I’ve double checked this with the network inspector in Chrome.

  • RyanMJones

    Go to https://www.google.com search “what is my referer” click. Google redirects you to a non-https version then sends a referer. Mobile isn’t doing that redirect. That’s the problem.

  • http://crem.in Jonathan Cremin

    I had to log out for this to work. When logged into my google account, the intermediate page is over SSL.

  • http://twitter.com/YoungbloodJoe Joe Youngblood

    Then signin to Google and get ready to lose a lot of benefits.

  • http://www.webkruscht.com/ Frank Zimper

    Nail on the head. All mobile searches on https search do not transfer any referrer to a plain http target site. I guess Google got rid of the intermediate redirect to speed things up. Mobile connections still tend to be somewhat slower.

  • http://searchengineland.com/ Danny Sullivan

    Well, Google decides that it is a publisher’s business if you search on Google and click on an ad. In that case, Google goes out of its way to tell the advertiser what you searched for and also doesn’t encrypt the referrer, exposing potentially private information to eavesdropping. And that’s not really a good thing.

  • http://searchengineland.com/ Danny Sullivan

    Actually, it’s iOS 6 in part doing it, because the behavior in Safari is now apparently to pass your searches through Google SSL search. That’s what’s leaving things empty. If it hadn’t made that change, referrers would pass.

  • http://searchengineland.com/ Danny Sullivan

    The https specs don’t come into play here. Everyone who has been trying to figure all this out based on them is making a mistake.

    Google doesn’t use the https specs for its Google SSL Search. If it did, then no referrers would pass at all, except if someone clicks to another secure server.

    But in Google SSL Search, Google users its own standard. Well, not standard. It’s just Google doing what it wants. It passes a referrer with search terms omitted if you click on a non-ad link. If you click on an ad link, a referrer with search terms left alone are passed. That you’re being redirected through Google and spat out with an unencrypted channel doesn’t matter. Google itself is deciding exactly what to allow into that referrer, not your browser or some https spec.

    That’s all on the desktop. It sounds like on mobile, from what you’ve spotted, Google SSL Search works differently (and probably by mistake, not how Google intended). It seems like they aren’t redirecting outbound clicks through Google, as they do on the desktop — right? And in that case, it means that since you’re going from a secure site to a non-secure site, the “normal” standard of not passing a referrer kicks in.

    It also sounds like no one’s really noticed this before because for the most part, most searches on mobile weren’t being done through Google SSL Search. Apple made this shift — and the “protection” Google promised for desktop search was never applied to mobile search — so suddenly, all that referrer data went poof.

    I suspect in short order, referrer data will be back — but not with search term data, so that this is all aligned with desktop search.

    And nice detective work, Ryan.

  • http://foliovision.com Alec Kinnear

    It’s not about privacy, it’s about sabotaging SEO work. At one point, Google will go directly into the SEO business. Since they will be the only ones with the data, it’s a virtual monopoly.

  • cjvannette

    Yep, before iOS 6 dropped, we had 68% google organic, 9% direct none. Now it’s 58% google organic, 19% direct none.

  • cjvannette

    Indeed. We’re trying to figure out a bounce rate spike and this means we can’t compare ACTUAL direct traffic and google organic traffic to see if there’s a difference there.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide