Post-PRISM, Google Confirms Quietly Moving To Make All Searches Secure, Except For Ad Clicks

Not ProvidedIn the past month, Google quietly made a change aimed at encrypting all search activity — except for clicks on ads. Google says this has been done to provide “extra protection” for searchers, and the company may be aiming to block NSA spying activity. Possibly, it’s a move to increase ad sales. Or both. Welcome to the confusing world of Google secure search.

Two Years Ago: Secure Searching For Logged-In Users

In October 2011, Google began encrypting searches for anyone who was logged into Google. The reason given was privacy. Google said it wanted to block anyone who might potentially be eavesdropping on a string of searches made by an individual and also prevent the actual search terms themselves from being seen by publishers, as some of them might be too “private” to reveal.

This Month: Secure Searching Being Made Default For Everyone

Now, Google has flipped on encryption for people who aren’t even signed-in. When asked about this last week, Google confirmed the shift, saying:

We added SSL encryption for our signed-in search users in 2011, as well as searches from the Chrome omnibox earlier this year. We’re now working to bring this extra protection to more users who are not signed in.

I sent a series of follow-up questions to Google after getting that statement and am still waiting for a response to them, so I’ll update as I hear more. Is this worldwide? How soon until it happens for everyone?

A Sudden Change

One key question is “Why so suddenly?,” what prompted Google to make such a change out of the blue. And it was sudden.

When searches are encrypted, search terms that are normally passed along to publishers after someone clicks on their links at Google get withheld. In Google Analytics, the actual term is replaced with a “Not Provided” notation.

Over the past two years, the percentage of search terms as “not provided” has increased as Mozilla’s Firefox in July 2012Apple’s Safari browser in iOS 6 in September 2012 and Google’s own Chrome browser in January 2013 have used encrypted search, even when people aren’t signed in at Google.

That’s lead to a steady increase but not giant leap in “not provided” activity. But in the past month, the increased encryption on Google’s side has produced a dramatic spike:

Not Provided Count - Charting the rise of (not provided) in Google Analytics-1

The chart above is from a site called Not Provided Count, and it tracks the percentage of terms being withheld across 60 different web sites. You can see the spike that began around the week of September 4 and which currently shows almost 75% of terms being withheld.

It was after viewing this chart on Friday that I asked Google if there had been some type of change, because the percentage of not provided terms can also vary from site to site — or even for “basket” of sites for different reasons. As noted, the change is real, and confirmed by Google.

There are two main reasons why Google may have made such a quick switch, and perhaps both are even factors.

Done To Block The NSA?

The first is the whole US National Security Agency spying thing. In June, Google was accused of cooperating to give the NSA instant and direct access to its search data through the PRISM spying program, something the company has strongly denied. That hasn’t saved it from criticism.

Since then, Google’s waged a campaign to allow it to be more transparent about the number of spying requests it says it does receive — on a limited basis and not involving direct access — but which it’s forbidden by US law from disclosing. It also began increasing encryption between its own data centers.

I suspect the increased encryption is related to Google’s NSA-pushback. It may also help ease pressure Google’s feeling from tiny players like Duck Duck Go making a “secure search” growth pitch to the media. Duck Duck Go and StartPage.com have seen large gains in traffic, though relatively speaking, what’s large is nothing for Google. The PR loss is far, far greater than the user loss, if any. But Google doesn’t like PR losses of any type.

Done To Boost Ad Sales?

The other reason is that Google recently made a change so that one of the easiest ways for publishers to see the actual terms that have been withheld over time is through the Google AdWords system.

See, apparently search terms aren’t so private that Google withholds them entirely. Rather, it withholds them from being transmitted in the clear across the internet. Publishers can still see these terms by going into the Google Webmaster Tools area, though they only see the top 2,000 per day and only going back for 90 days (something Google said earlier this month will increase to one year, in the future).

If publishers don’t somehow constantly archive these terms, they’re lost. But a change to Google AdWords in August allows publishers to store the terms as long as they like, for easy and instant access — as long as they use Google’s ad system.

It’s an odd situation that Google won’t archive search term data within the toolset it expressly built for non-advertisers — Google Webmaster Tools — but does allow this through its ad system. It suggests that terms have been withheld all along in part to create new Google advertisers.

Privacy Loophole Remains For Advertisers

That’s especially so given that ad search traffic has never been made secure. No encryption stops people from eavesdropping on the terms used when someone searches at Google and clicks on an ad. Google’s also never prevented this information from flowing directly to advertisers, in the way it has for non-advertisers.

More Background

So. Increased privacy to thwart the NSA? Or a handy excuse to do that and increase potential ad sales? If I learn more, I’ll update. Meanwhile, for more background, see the key article from earlier this month, below:

Postscript (5:05pm ET):

Google’s sent this update:

We want to provide SSL protection to as many users as we can, in as many regions as we can — we added non-signed-in Chrome omnibox searches earlier this year, and more recently other users who aren’t signed in. We’re going to continue expanding our use of SSL in our services because we believe it’s a good thing for users….

The motivation here is not to drive the ads side — it’s for our search users.

It’s still not clear why Google seems to have ramped up things especially today.

Related Topics: Channel: SEO | Features: Analysis | Google: AdWords | Google: Analytics | Google: Webmaster Central | Legal: Privacy | Top News

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Christian Noel

    If they stick to the “privacy” line as far as a rationale I don’t see a scenario where they then turn around and start selling Keyword data. Even if they did I wouldn’t buy it. There are other methods for gleaning KW data without buying from Google.

  • http://thejakejordan.com/ baldjake

    I think you meant Mwuahaahahah!

  • http://nickpierno.com/ Nick Pierno

    Exactly. If you could only get the data by using secured servers then that would encourage most sites to go in that direction. Internet becomes safer, SEO’s get keyword data… everyone wins.

  • http://www.geekstogo.com Blair Briggs

    Prism and NSA talk is a red herring. The NSA already has access to Google’s SSL master encryption key, so they can decrypt both the search data, and data flowing between data centers.

  • Paul Jewkes

    I think this is fair enough and slow enough warning-
    Nov- 13%
    Dec- 14.5%
    Jan- 25%
    Feb- 28%
    March- 32%
    April- 26%
    May- 30%
    June- 32%
    July- 31%
    Aug- 45%
    Sept- 64%

  • Steve

    Google will probably start monetizing this data also. It’s becoming more and more about profits for Google.

  • http://www.saturngames.co.uk/ Dan Crocker

    Hopefully not

  • Jeremy

    Why would Google eliminate a metric within their own analytics tool? So forever and always when we go into Traffic Sources -> Sources -> Organic : we’re going to (eventually) see Not Provided at 100%? Seems a little barbaric for Google to pull that on everyone. SEO’s or not!

  • joeyoungblood

    If they do that, I’ll apologize for railing against them the last few years. I’ll also shave my head, and donate a weeks worth of pay to the charity of their choosing. I doubt it will happen though.

  • MikeInEdmonton

    Hmm I wonder what they are up to? I think it’s pretty obvious, the NSA part of the story is a diversion in my opinion to add credibility to the claim. I believe this “not provided” data will be sold back to us in the very near future, or at least I hope so. I need this data to make informed decisions with our marketing goals.

    How many think it’s a ploy to make it unstable for an SEO to wean a client off of their Adwords spend? I have heard a lot of chatter with my clients recently regarding the effectiveness of their Adwords spend and making that transition to an SEO solution is going to be a tough sell. Does that sound like something Google would do?

  • Daniel Benny Simanjuntak

    I was so expecting that. Google wants to generate more and more money from Adwords which means they need to make seo difficult so that companies can move to ppc to get results.

  • http://www.dirigodev.com/ David Addison

    If you have Hitwise or Comscore (clickstream data) you can get at the data they’ve blocked. So how are they creating a more secure internet when ISPs sell access to data? I presume that this is where the NSA gets data. Here in Portland Maine the Department of Homeland Security rents space inside the old Verizon building on top of the phone network on Cumberland Avenue. They’re not making the internet more secure. They’re hurting marketers and solidifying the position of Google Analytics.

  • Joel Solomon

    As a note, I confirmed with seoClarity that (if provided access to Webmaster Tools), they can archive keyword performance.
    This is regarding your line “If publishers don’t somehow constantly archive these terms, they’re lost.”

  • http://www.blakestrategiesgroup.com/ Jonathon Hyjek

    This is not helpful, but it will serve to increase Adwords spending. If G can convince website owners that in order to get good quality data, one has to use their very own PPC product. It’s like a license to print money.

  • http://searchengineland.com/ Danny Sullivan

    Yes, several third-party tools can do this. So can AdWords. The point is that if AdWords can do it, for free, then the tool designed expressly for publishers by Google should also be able to do it.

  • http://searchengineland.com/ Danny Sullivan

    They haven’t said this is due to PRISM. I suspect that might have prompted it. But regardless, the official line is that this has all been done to increase privacy. And that’s true. It does. But it also leaves a big loophole for advertisers, which opens the door that they are also doing it for other reasons.

  • BJRCollins

    Is it not allowed to be both? A business’ sole responsibility is to make a profit for its employees and owners. If there is no profit, there is no company. Assume that this was done 100% profit motive. Are you not still more secure now than you were before? How many of you have switched services/stopped using Google?

    Just because a person or business does something motivated by survival does not mean that it is bad for the rest. Unlike the government, there are alternatives. If a business is not providing what the customer wants, they may go elsewhere. So motivation to act against the consumer is not in anyone’s best interest.

  • BJRCollins

    In doing so, this would create not only a more secure environment for all, it would create organic real growth in the industry and economy. Google makes more money. Security keeps up with the times. Jobs are created and GDP increases.

    Win-win for a move being derided by those who have the most to gain. Optimists and Pessimists dither while Realists move forward and prosper.

  • BJRCollins

    I wish I could upvote x10.

  • BJRCollins

    Doesn’t stop the fact that the people are stupid and Google is a publicly traded company. Let’s not put any lipstick on this pig and call it like it is – a move made to mollify the ignorant masses before they can destroy a good company – one that has made the lot of you a great deal of money.

  • pojda

    BS, NSA can break SSL easily… I’m more inclined to “Done To Boost Ad Sales”

  • http://www.eemes.com/ Eemes

    End of SEO Keywords! This is the worst scenario for SEO Experts!

  • http://www.eemes.com/ Eemes

    True, this gona hurt a lot. DAMN!

  • Craymin

    Smoke and mirrors… NSA conducted man-in-the-middle attacks to snoop on Google users SSL traffic (ref. NSA Flying Pig). If Google were serious about privacy the first thing they would do is stop logging, stop complying with warrantless requests and start contesting the Feds demands for end user data. Warrants issued by a FISA court in which the defendants (aka American Citizens) have no representation is an abomination of the 4th amendment. And lets not forget it was Google’s own lawyers who said GMail users have “no legitimate expectation of privacy in information”.

  • Paul Jewkes

    I wont argue that… but I will argue that it’s a bad thing.

  • http://www.eBizROI.com Rick Noel, eBiz ROI, Inc.

    First the retirement of the external keywords tool. Now the rapid and unannounced acceleration of encryption of all organic searches. It certainly seems like a Google push to move SEO investments to PPC.

    If Google has an aversion to bad PR, this move will create friction at least the online marketing industry. Regardless of the actual motivations, perception matters.

    Realistically, with Google’s dominant market share and no comparable index, what’s the risk to Google? Will the average user notice or care? Probably not. Will webmasters block googlebot. Some, perhaps, but that is like cutting off the nose to spite the face.

    Until there is a comparable search alternative, or the governments pursue anti-competitive litigation, we will all have to adapt and get the data from other sources. It sucks, but it just is.

  • http://blackhatpwnage.com/ igl00

    since google is part of PRISM the only thign they hide is keywords form US not from the US.GOV

  • Jacob Maslow

    Google can’t take an action to explicitly block the NSA. That would probably be illegal as it can be interpreted as trying to circumvent a court order. If they rush on something they have been planning and is “good for users”, that’s fine even if it happens to make the NSA job harder.

    The NSA is seizing search records for thousands and possibly millions of users. Now the NSA isn’t getting something they can upload right to their data center so their contractors can stalk their love interests, least favorite politician/celebrity or do stupid keyword searches across a million accounts.

    The NSA should have to go through hoops.

    If you weren’t an American, how comfortable would you be using google? Right now, google is extremely vulnerable overseas to companies like duckduckgo. If a local search engine does a major out reach campaign emphasizing privacy, they can hurt google in their market.

  • Jacob Maslow

    If a judge orders me to testify, I may decide to go through with my vacation plans, surgery / rehab. As long as I can argue it’s something I likely would have done anyways.

    I don’t think google’s lawyers will allow them to do anything specifically against prism without a strong business necessity justification.

    Even if it is legal and doesn’t violate the spirit of any court order, a judge likely can order them to cease any anti-prism measures.

    Expecting google to tell you that they are doing something to fight the NSA is not realistic. Google has a right to be opposed to the NSA and Google can speed up anything they had in the works, but google likely can’t do anything solely to impede the NSA. Even if their real reason is that prism is bad for business, broadcasting it would be stupid.

  • http://withnoble.com/ Bryant Lack Jaquez

    Crazy, crazy world of search!

  • Navjot Singh

    I think Google wants to generate more and more money
    from Adwords which means they need to make SEO difficult so that
    companies can move to PPC to get results.

  • Marcel Wengel

    Is there an official statement regarding this stuff from google so far?

  • Rajesh

    Danny, then why do you use “Post Prism” as the first two words of your title?!!!

  • Alistair Dent

    Hi Danny,

    It’s not that the organic searches are encrypted and the ads aren’t, the entire page that is served to the user is encrypted.

    Depending where you click something else happens. In both cases the user goes through an insecure redirect. This is so that the browser can still detect that the traffic came from Google.

    In the case of an organic click that redirect contains next to no information. In the case of an ad click it contains the query string (gclid) that has been appended to the ad. Since this gclid is meaningless outside Google’s systems there is no privacy issue with passing that across.

    GA and AdWords can parse that back into a keyword internally, but no keyword information has been passed to the publisher, nor could anybody intercepting that traffic extract a keyword from it, the way they could do in the old days with organic searches.

  • Rajesh

    It is like saying if you pay, you get access to the “privacy” data! so NSA is anyway going to have access and “Post Prism” is no way relevant or related to this.Isn’t it?

  • Navjot Singh

    Google wants to generate more and more money
    from Adwords which means they need to make SEO difficult so that
    companies can move to PPC to get results.

  • Dave

    12 days ago I wrote this on Seroundtable:
    Future!!! I am now redirected to https://www.google.com, even when I am not signed in, not only in Chrome, but in all browsers I can think of IE, FireFox, Opera, Safari etc.

    Is this the future???

    Seems like this ->NOT PROVIDED is the future.

  • http://benacheson.com Ben Acheson

    There is only solution to Google’s Not Provided scam.

    Big brands need to #BoycottAdwords

    https://twitter.com/BenAcheson/status/382432178224652288

  • Dave

    No, its not an error. Google pushed SSL search in the first week of Sept and this has been the reason in increase in NP

  • Dave

    Real world effects- NOT to USE Any ANALYTICS for tracking Google ORGANIC Traffics.

  • Dave

    And what about Adsense ads on HTTPS websites?

  • http://www.barryadams.co.uk/ Barry Adams

    The PRISM connection you’re making is entirely arbitrary, Danny. You might as well say this is a post-Fukushima move, for all the relevance it has.

    Google still has all the keyword data itself – the fact it gives it freely to AdWords advertisers proves that. They just decided not to share it any longer with website owners.

    The NSA, by virtue of their backdoor access to Google’s data centres, can still spy on that keyword data at their leisure. So to attempt to present this as a PRISM-motivated move is farcical and serves only to give the impression Google has done this for the benefit of its users (which is a blatant lie).

    A serious technology journalist would therefore never make that PRISM connection – which is also why Google has not pushed that angle, as it’s so easily falsifiable. You are once again serving as Google’s voluntary PR spin machine, Danny.

  • Dave

    Right said..I did commented as a comment reply where an expert is happy with NP keywords. This is what i said to him.

    For you “not provided” doesn’t matter. Means you are happy, if people
    are coming to your site with some sort of keywords and your bounce rate
    is 100% and the keyword is shown as “not provided” you would still be
    happy.

    Keywords are important for on-page experience. People may be coming to a website with a keyword which have low relevance with the content and are bouncing from the page. With keywords data, we can analyze which one are best as per our business and services and adjust on-page accordingly

  • aline

    OMGGGGGG…Whats going on?!

  • jonathanwthomas

    Well, screw you Google. Guess I have no right to know how people arrive at my website.

  • nextgreencar

    Yes my thought exactly – a competitive move against “search retargeting” companies

  • wwd88888

    The NSA is already INSIDE the Google data centers, just like they have NSA rooms at all the carriers and ISPs. They are using IETF RFC 3924 the lawful intercept architecture built into the network switches at every major ISP and cloud provider.

    They don’t need to “crack SSL”.

  • Vero Tabares

    Why give publishers and potential advertisers ACTUAL traffic volume for keywords when Google can imply which keywords are receiving traffic (for a price)?

  • Francisco Bustamante

    Should we just now solve our keyword questions checking on Webmasters Tools?

  • David McCarthy

    The majority of our clients are micro- businesses … quite frequently single-person businesses. They don’t have the time or money resources to correlate the changes they make to their websites with any change in traffic, and where the traffic is low volume, there won’t be any statistically significant correlation. They will be losing out because they will have a much poorer understanding of how their potential customers are trying to find them … and consequently, the customers will lose out too.
    Only companies with a significant turnover will be able to afford to work on this. Recent research in the UK suggests a huge proportion of these micro-businesses have an annual turnover of £18,000 or less.
    Only Google will benefit as more businesses feel their only option is to use PPC.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide