Google’s all-new, quite counterintuitive guide to ad testing

Over 2 1/2 years ago, I wrote about how ad rotation works in AdWords. Since then, a whole lot has changed. Most recently, we’ve simplified the options for ad rotation. Let’s take a closer look at how ad rotation works now and what that means for ad testing.

How AdWords picks which ads to show

The goal of ad-serving in AdWords is to deliver a tailored message that meets a searcher’s needs. This means delivering ads that people want to click on and getting better results for your business.

In the new world of ad rotation, things are pretty straightforward. You have two options:

• Optimize: Prefer best-performing ads.
• Do not optimize: Rotate ads indefinitely.

When you don’t optimize and rotate your ads indefinitely, the system will rotate ads from the ad group to choose which one enters the auction. If it’s that ad’s turn, then it’ll enter into whatever auction is happening at the time. A worse-performing ad will get its turn in that auction as long as you leave that ad running.

Optimized rotation — which both Google (my employer) and I recommend — considers a bunch of signals, none of which is “which ad’s turn is it?” I’m a firm believer in the optimize setting, and I think you might be one, too, after hearing how it works.

AdWords takes many things into account when predicting your CTR (click-through rate), such as the user’s query/device/location, the historical performance of your ad and factors which affect the visibility of your ad, such as position and extensions.

When you think about that pairing between ad and query, I think it’s easy to see why we also recommend more ads whenever possible. When you have more ads present in an ad group, you increase your chances of finding the right match across all of those variables we look at.

That’s the theory of it, and our internal numbers show the performance benefits are real. Going from one ad to a minimum of three ads in an ad group can give that ad group up to a 5 percent to 15 percent lift on average in both impressions and clicks (with incremental uplift for each additional creative).

Ad rotation in action

Let’s see some examples of ads in action. Imagine you’re advertising a hotel trying to generate bookings in New York. One of the keywords you’re bidding on is “hotels in New York.” From that one hard-working keyword, you’re matching to queries like “best hotels in NYC,” “cheap NYC hotels,” “new york hotels central park” and “hotel in new york tonight.”

For this simple example, I’m referring to a broad match keyword that can match to different queries. I don’t want to tackle account structure guidance here, but this advice about more ads applies even if you prefer more specific ad groups with a tighter match between keyword and ad. If you want to talk account structure, you can check out my previous post that touches on the problem of oversegmentation.

Anyway, here’s the ad you have in your ad group:

On the query “hotel in new york tonight,” this is a fantastic ad. Based on everything at play, that ad could show up in position one with an outstanding CTR. However, performance would be predicted to be much worse on queries that don’t mention booking for tonight. If a user is worried about saving some money or being near Central Park, the ad above doesn’t have a ton to offer. As such, you might lose impressions on those queries.

After looking at your search terms and performance overall, you decide to add some new ads to the ad group with new headlines. Here are the ads you implement:

You now have an ad group that is much more competitive on additional queries (“cheap NYC hotels” and “new york hotels central park”). You also might start to see additional impressions on new types of queries like “discount hotels new york.”

Here’s one thing to consider: You’re winning more impressions on those queries, but you might be branching out to areas where the competition is doing a great job. On those additional impressions you’re receiving, your position on the page is worse than what you’re used to. A lower position translates to a lower CTR on your ads. But you’re now competitive in more auctions, getting more impressions and clicks and reaching queries you might not have been able to before.

Don’t make decisions based on ad-level metrics (including ad-level CTR)

One ad’s CTR shouldn’t be the main way to decide how effective that ad is. Here’s what I’d propose instead: Use ad group-level metrics, particularly impressions, clicks, conversions and CTR. Those metrics are much more impactful for your bottom line than one ad’s overall CTR would ever be. Ad group-level impression share is another great thing to start reviewing before, during and after ad tests. (Ad-level impression share doesn’t exist for a variety of reasons.)

CTR alone can be misleading because ads show on all sorts of queries in all sorts of contexts. Different ads in the same ad group will show under totally different circumstances; there’s no way to control for all of the different devices, locations, situations and everything else that goes into one auction.

Think about the hypothetical Central Park ad I mentioned before. It might be the lowest CTR in your ad group, but if you pause the ad, those impressions aren’t being redistributed to another ad with a higher CTR. Those impressions are simply going away.

But I don’t want to stop testing my ads!

I like testing, too! I’m not suggesting it goes away entirely. I am, however, suggesting that you remove A/B ad testing from your standard operating procedure. Here’s a test I’d recommend if you really need to scratch your testing itch.

Test an ad group with one ad (A) against an experiment ad group with four ads (A, B, C and D) with rotation set to optimized. You can use drafts and experiments to create these two versions. That way, you’re testing to see whether or not more ads result in more impressions and clicks at the ad group level.

Keep in mind, though, that the whole point of the optimized setting is that you don’t need to test ads A and B against each other anymore. Those two ads now work together instead. As a result, you don’t need to discard ad text that has a “losing CTR.” Instead of choosing the winning option A and making the losing option B leave town, you should plan on having options A, B, C and D all active at any given time. Delete stuff whenever an ad stops seeing a large fraction of the impressions and therefore generates minimal to no clicks. Then add a new ad to the mix. It’s better to have options.

And here’s a quick note: No matter if you are using manual bidding with optimized rotation or Smart Bidding, our system is always working to find the best creative to serve. For the selected creative, Smart Bidding adjusts the bid based on the predicted conversion value. No matter what bidding you use, my advice about ad testing remains the same.

Conclusion

Ad rotation has been streamlined recently, but selecting which ad will receive an impression is a more involved process than it’s ever been. Add more ads to your ad group so that your optimized ad rotation can win more impressions, clicks and conversions for your account.