• James Davis

    Thanks for this, Danny. We’ve been struggling with this problem and it only seems to be getting worse. For the last 30 days, 65% of all traffic in Analytics for our main site is “not set” or “not provided.” In WMT, Google reports we took 2500 clicks for the same period, but only reports 400 of those clicks – ALL of which were spread across just 2 branded keywords. This means that the data Google serves us is almost useless. Unfortunately, the problem is that although we have at times spent as much as $30,000 per month in Adwords, the data is not the same and cannot be used the same way. When you develop Adwords ads, the text, keywords, message, landing page and action are all different than what is targeted and setup organically. So while there is valuable data to be mined with a decent spend in Adwords, it doesn’t correspond – at least in our industry – with organic data at all.

  • Tom Roberts

    “I don’t think Google planned for this to happen. But that’s the end result”

    Come on Danny. I don’t believe for one second you’d be this naive, you’re too skart of a cookie. Of course it was deliberate.

  • http://www.seoconsultant.ie SEO Consultant

    The problem is if you use Marketo / Salesforce – to store your search queries. There you still get ‘Not Provided’, since you can not get Marketo or any similar tool to log into AdWords. :(

  • John Broadbent

    Well I guess we need to prepare for everything.

  • joeyoungblood

    Hooray!!!!!!!!!!!!!! I’ve been saying this since they launcehd NP. It’s a move to make more $$ in the long run and knock out ‘search retargeting’ services

  • joeyoungblood

    Unfortunately I agre. Since Larry Page took back CEO control and Marissa Meyers was moved to Maps/Local the blurring of user experience / privacy and making Google more long-run money has gotten exponentially worse. Google is a public company and making small changes little bits at a time to adjust behavior is how they can make more long-run money for another decade to come, unless they start losing market share.

  • joeyoungblood

    “not set” I believe is PPC traffic being incorrectly assigned, check your tagging. https://support.google.com/analytics/answer/2820717?hl=en

  • Colin Guidi

    yea that’s correct

  • Colin Guidi

    I want to link the two properties, but I’m on the fence. Hypothesizing what Google might do when SEOs open the fire hose from WMTs to AdWords.

    “Google is forcing publishers into its ad interface to get information about their “non-paid” listings”

    I mean, they already have this data, but what will happen when we readily make it available through linking the two? This, I’m not sure about.

  • joeyoungblood

    Google is also taking what is left of keywords with their new “local carousel”. When you click on a business image/name Google runs a new search query for their brand name. This would pass on branded keywords to the site instead of the actual search query.

    I think this helps add to the evidence that Google is not doing this haphazardly or accidentaly.

  • jnffarrell1

    Google’s action was in-Advertisment. It makes sense. For historical purposes Advertisers can journal their own historical data by storing today’s result for comparison next year. Moreover, 90 day data access may be part of negotiations with the EU. Furthermore, data encryption is coming to Google’s cloud and advertisers will have access to only the last 90 days.

  • Catfish Comstock

    If it isn’t too big of a concern to apply to paid traffic, it shouldn’t be too big to apply to organic. Otherwise its exactly what it sounds like it is. If they would at least make a brand / non brand filter for your Keyword Unavailable that you could see in Webmaster tools you could at least know where your traffic was coming from. Right now, it’s impossible for businesses to scale because they don’t know if the uptick or decrease in traffic is due to their brand being more or less popular or if they actually have improved their footprint across relevant non brand traffic. This whole thing is really ridiculous and Google has yet to prove that what they did was really all that necessary (but not necessary enough for paid searches). I mean if you think about it, it’s a pretty indefensible position that reeks of greed and manipulation (which is how most people feel about the whole thing).

  • http://www.instantatlas.com/ David E Carey

    ‘Duh! Really do you dink sooo?’ Never thought for a moment that this strategy by Google was really anything to do with ‘privacy’ of the searcher. I find myself (as are my competitors) having to some degree introduce PPC into the online marketing mix. Funny how I can see many more keyword phrases reported in my analytics for leads that originate via the PPC channel – mmmmm!

  • Tom

    The lack of keyword referral data also results in a worse user experience for the search by preventing personalisation on the resulting website based on the search term. If Google really believe this is an issue if user privacy they should only pass referrer data to SSL sites. Within months a large proportion if the web would move to SSL, also helping Google in their new found battle to stop the NSA tracking everything and everyone ( http://g.re.af/gnsa ).

  • kimberly537

    My Uncle Julian just got a new white Land Rover Range Rover only from part-time off a computer. you can try this out w­w­w.J­A­M­20.c­o­m

  • Dave

    For the past few days, I have been redirected to https version of Google search, irrespective of what country I searched for, even though I am not signed in. This is happening for IE, Firefox and Chrome. This mean more “not provided”, if it’s going to be implemented internationally.

  • Scott Davis

    (not set) is direct traffic and did not come through with a keyword by definition.

    I feel your pain though. I’m dealing with 52% (not provided) myself.

  • Colin Guidi

    That’s not true. (not set) can come from several different reasons, but the most common (from what I’ve seen) is when you don’t properly tag your PPC campaigns. For example if you run in Bing and don’t manually tag your URLs.

    Go to joeyoungbloods link above, it’ll explain it for you.

    And direct traffic is traffic directly driven to your website, therefore users bypass SERPs and do not enter keywords but rather your web address, so of course there wouldn’t be keywords associated with it.

  • Scott Davis

    If you go into the keyword view under direct traffic, you’ll see that all of the keyword information is (not set).

    You’ll have to do this by changing the primary dimension to Traffic Source -> Keywords

  • http://searchengineland.com/ Danny Sullivan

    For various reasons, I don’t think it was planned this way, as I explained. But as I also explained, it doesn’t really matter. Planned or not, the end result is that it might as well have been planned. But I did drop “inadvertent” from the headline to punch that up a bit more, and put the emphasis back on the end result.

  • Tom Roberts

    And I just realised I tried to spell “smart” like “skart”.

    But thanks for the follow up Danny. You’re definitely a smart cookie, not a cookie made from a skart/scart lead.

  • Matt O’Toole

    Having a brand/non-brand filter wouldn’t work perfectly as you’d be entering brand terms blind, without knowing if someone has mis-typed or otherwise used a branded phrase you just hadn’t thought of; you’d have to add in an almost endless list of branded term possibilities.

  • http://www.archology.com/ Jenny Halasz

    I also think it was accidental at first, but that as time goes on, it becomes more and more about increasing the bottom line. Not provided, combined ad/organic data and the recent changes to the keyword tool all point to one thing… increase the number of advertisers and the number of keywords they are bidding on. Sorry to link drop, but we covered both sides of this issue on the Archology blog last week… you might be interested in reading: http://www.archology.com/organic-data-and-not-provided/

  • PeterD

    If privacy was the “main concern”, then why isn’t Adwords referral data blocked, too?

    And are we to believe Google just “stumble about” making these significant changes without realizing the consequences? Really?

    I think they know their business very well indeed.

  • Spreading Wisom

    Nobody in their right mind would believe that this was “accidental”. Not buying that “privacy” BS either… queries in analytics are anonymous. It’s not like there’s an IP address attached to the keyword or a Facebook account.

  • Spreading Wisom

    Nobody in their right mind would believe that this was “accidental”. Not buying that “privacy” BS either… queries in analytics are anonymous. It’s not like there’s an IP address attached to the keyword or a Facebook account.

  • Garrett Rent

    So, when does the paid version of GA (with “enhanced” keyword visibility) release? (Over 10,000,000 sites using GA) x ($19.99/mo.) = serious overnight incremental revenue stream. And, with GA you hook all the SEOs and SMBs that don’t wish to (or can’t afford to) play w/ Adwords each month.

  • http://searchengineland.com/ Danny Sullivan

    Heh. I have some old scart leads in my junk box. HDMI is so much better :)

  • Cathy Dunham

    I first noticed this dramatic drop in one of our client’s organic keyword totals two days ago… so I’ve been searching diligently for some solid, credible answers that explain why this occurred. Thanks, Danny and Matt, for filling us in. However, let me add my two cents:

    Dear Matt Cutts,

    The hardest part to understand is that Google has increasingly promoted “writing quality content that’s relevant to your users”. I totally agree! And, I’ve been using those keyword search terms for years to help me define the more specific details of content my clients’ search visitors want to find. We’re just trying to do our best to provide a richer user experience (in total agreement with what Matt Cutts has been saying). And now you chop off our most valuable resource tool? DUH! And not to mention how totally painful and useless the Keyword Planner Tool is now compared to last year! I’m usually a mild-mannered speaker who rarely swears, but WTF?

    Affectionately Google-friendly Optimization Strategist,
    Cathy Dunham

  • natfinn

    We’ve always told our clients two things when it comes to Google:
    1) follow the money in regards to their product updates &
    2) for real keyword data, trust a CPC campaign’s information. We never thought it would really come to this.

    #DarkGoogle has really squeezed the sales funnel this time.

  • Catfish Comstock

    I didn’t say it would work perfectly but at least a trend against typical brand terms like company name, product names would be a good start as these terms are the brand terms that get most of the traffic. Thus eliminating traffic numbers for those terms would at least give you an 80 to 90% view of your brand traffic and would allow you to make year over year comparisons that were more meaningful in terms of understanding of you have improved your SEO performance.

  • Catfish Comstock

    As soon as shareholders figure out that revenue stream you can count on it. Unfortunately Google going public was the end of do no evil.

  • Catfish Comstock

    As soon as shareholders figure out that revenue stream you can count on it. Unfortunately Google going public was the end of do no evil.