How much does Google’s new search term filtering affect ad spend transparency? Here’s how to find out

Google announced at the start of September it would reduce the visibility of search terms shown in reports for advertisers. Privacy was the reason cited by Google but advertisers quickly called the motives into question and complained about the loss of transparency into a large amount of ad spend. 

A few well respected PPC influencers weighed in on the impact of this change on Twitter.

Wonder how much visibility into search terms we lost? We went from 98% visibility down to 78%….overnight.

This is tens of thousands of queries we would have mined for inefficient spend that we now do not have access to.

Who pays for that? Advertisers.

— Wil Reynolds (@wilreynolds) September 2, 2020

Hmmm, Google already had a filter set to not pass all Search Terms into the UI based on traffic volume, so this appears to be them simply moving the threshold. On the other hand, it is yet another in a progressing series of changes to remove data from external access. #ppcchat https://t.co/WjCBd9TXxi

— Kirk Williams (@PPCKirk) September 2, 2020

Ultimately, what really matters is the impact on your own accounts and whether the change limits your ability to effectively optimize things.

A script to calculate how many queries are unidentified

So we at Optmyzr wrote a script to help you to see the impact for yourself.

The script pulls daily metrics for as many days back as you’d like. You can choose to focus on impressions, clicks, or cost. It pulls the data two ways:

1. Using an account report, filtered for Ad Network Type = Search
2. Using a search query report, also filtered for Ad Network Type = Search

It then compares the key metric, like cost, between the two reports. The account performance report should include data from all search queries, whereas the search query report now only includes data for search queries that made it past Google’s new filters.

We then use these two numbers to calculate the percentage of unidentified search queries — column D in the example below.

The search query report metrics in column C are lower because of Google’s search term filtering.

Finally, we plot this percentage of clicks going to unknown search terms by day, allowing you to easily see if there are any big changes coming out of the Googleplex.

Findings

The point of sharing this script is to make it easy for everyone to see the impact of this change on their own accounts. I’ve run it on only a handful of accounts myself, so my findings are based on a small sample set.

Here’s what I found:

1. Impressions were already filtered a lot before Google’s announcement.
2. The change took effect September 2, 2020, the day of Google’s announcement.
3. There is a pronounced decrease in visibility into queries driving clicks and cost.
4. E-commerce advertisers who use Smart Shopping campaigns saw less of a change because most search terms were already unavailable to them.

Let me illustrate the large amount of filtering that’s been happening for quite some time first. 

Here’s a chart for the level of filtering Google does when we consider impressions as the key metric.

Notice that there’s barely a noticeable difference on September 2nd. More importantly, notice that more than 70% of my impression volume comes from search terms that are not in the search query performance report. This data goes back to mid July for this chart but you can run it for a longer period of time if you’re curious.

Next, let’s look at what happened on September 2nd with clicks and cost data, the metrics advertisers are justifiably more sensitive to.

Here’s a chart going back to mid-July based on cost for an account.

It’s clear that there’s a big spike in unidentified search terms starting September 2nd. In this particular example, the percentage went from the mid twenties to around 50%. 

This means that for this particular account, we used to not know which search queries were responsible for about 20% of costs. Now with the recent change, we don’t know which queries are responsible for nearly half of our ad spend.

The script

Ready to do the analysis yourself? Just copy-and-paste this script into the script section of Google Ads.

The settings are pretty self-explanatory. Put your email address on line 8. Line 7 and 9 can also be changed but the script will run fine if you leave these as they are.

Seeing interesting results? Let me know via Twitter.