Varian Shares AdWords Secrets – Will He Bring Transparency To Google?

I’m a big fan of Google’s Chief Economist, Hal Varian. When Dr. Varian speaks we gain authoritative and important new information about how the Google AdWords auction really works. Information that we had no way of gaining on our own but that significantly improves our ability to manage our accounts and campaigns.

I also like him because there’s a Joe Biden-like quality to the way Dr. Varian delivers the truth (and I mean that as a genuine compliment.) His communications are based first in facts and only secondarily packaged and limited to fit the party line. The result is that along the way we learn, or have confirmed, a truth or perspective that the PR flacks and marketing spinners would probably rather have remained unspoken.

Dr. Varian’s first major communication was the ‘How AdWords Auction Works’ YouTube video. It’s the “Subterranean Homesick Blues” of AdWords videos—a long complex stream of previously unknown truths which demands repeated viewings.

YouTube Preview Image

In it, Dr. Varian explained the math behind the calculation of both position and CPC in AdWords. Suddenly we could quantify the economic cost or benefit of Quality Scores and Quality Score changes.

As Paris Hilton would say if she ever spoke without irony, “that’s huge.”

Now in a post on the AdWords Blog Dr. Varian and his team report that they have verified that conversion rate doesn’t vary much based on ad position. This is critical information in our quest to intelligently set bids and build bidding strategies. Calculating the “right” bid without knowing that conversion rate was relatively constant was difficult, to put it mildly.

If conversion rate is relatively constant, then revenue per click is relatively constant (we’ll assume for the moment that basket size isn’t varying by position either). This means that if we can determine the variation in cost-per-click at different positions we can rather easily decide the “right” bid to maximize revenue or profit.

All of this means that Dr. Varian eviscerated one of the core assumptions about PPC bidding—that conversion rate varies dramatically by position—which was held by a lot of smart people and around which many an algorithm has been written.

To be fair, there are more than a few people who remain skeptical of Dr. Varian’s claim. I’ve seen blog posts and spoken to some smart and experienced paid search practitioners who believe they’ve seen data which contradicts his point.

Is Dr. Varian wrong? Are his claims too general? Are there marketplaces or situations in which conversion rate isn’t linear? Or was he able to hold the variables more constant, in terms of statistical analysis, than mere mortal paid search advertisers can because he has access to more and purer data? We don’t really know, but until proven otherwise I’m giving Dr. Varian the benefit of the doubt.

A new twist

But while the first impression is that Google’s release of this statistical analysis makes bidding easier, an aside Dr. Varian made as part of his explanation muddies the water again.

It’s in the comment where he explains that sometimes when you raise your bid your average position goes down. He says that as your bid increases the keyword you’re bidding on becomes eligible for new auctions. And in these new auctions, which you’ve just barely bid high enough to earn an ad rank that made you eligible, you’re likely to appear in lower positions, thereby lowering your overall average.

In other words, when you increase your bid, your ads should generally rank higher within the auctions where they were already showing—this is probably your intention when you raised your bid. But at the same time, a possibly positive but probably unintended consequence occurs: those same keywords with their new higher bids now trigger impressions for queries that they failed to trigger before. And for these new queries your ads are generally appearing in lower positions.

All of this is perfectly reasonable. The fact that the higher bid expands the range of search query eligibility is in line with what we know about the auction process, even if it’s a result many of us hadn’t considered. The problem is that it’s another example of AdWords giving us data that conceals what is actually happening. If you just looked at the data and made a reasoned analysis of it you would very likely draw the wrong conclusion. With a full understanding of the situation you’d know you really can’t draw any conclusion at all.

We’re buying advertising, getting data back which is ostensibly provided so that we can learn from our results and make rational decisions about improving our campaigns. And yet it turns out the data is inadequate and inconclusive. This isn’t the only situation where this is true, but it is a great example of how AdWords gives the impression of transparency while failing to provide the data we need to make smart optimization decisions.

Here’s what we now know:

  • We bid on keywords and review reports showing our clicks, cost, revenue, and position.
  • Along the way, we may raise or lower a bid to see the impact of that change on our results.
  • The bid change not only affects our performance in auctions we were already entered into, but it also makes our keyword eligible for more or fewer auctions. But we have no way of knowing about our participation in these additional auctions – Google doesn’t offer an ‘auction count’ metric.
  • The average position metric isn’t the indicator of impact we’d normally expect. When average position goes up, down, or stays the same after a bid change we have no way to correlate the two events. A basic assumption most of us had about bidding and reporting is shattered.
  • For the same reason, we can’t draw any particular conclusions about resulting changes in our click volume or average CPC. Our bid change might be having the exact result we wanted – rising positions, more clicks, higher CPCs, and increased net revenue – when run against the same queries we were always matching, while at the same time exposing us to a whole new set of queries which turn out to be poorly converting wastes of money that lower our numbers to the point where it may be impossible to tell that our bid change largely had the desired effect but unfortunately came bundled with a horrible side effect.

Of course, we can simply look at the net result, conclude that our bid change was a bad idea, and reverse it.

But that’s not the way many of us believe we should have to manage our keywords or accounts. We’d like data to be provided at a high enough resolution so that we can make precise and deliberate decisions. If there are multiple auctions going on in relation to a single keyword, and we can make money in some but lose money in others, we’d like the data and options that would allow us to choose where and how we participate.

The fog is lifting

AdWords advertisers have spent a lot of time and money feeling around in the dark. The information Dr. Varian shared in his video and his blog post are very helpful points of light. The new ‘Bid Simulator’ in the new AdWords interface is another.

It’s great to see Google sharing this information, and providing these tools. I look forward to this trend continuing to the point where paid search advertising on AdWords is truly a fair and transparent—although still competitive—marketplace. I think that’s where Hal Varian is taking us.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: SEM | Features: Analysis | Google: AdWords | Google: Employees


About The Author: is author of Quality Score in High Resolution and founder and president of ClickEquations Inc., which provides the ClickEquations paid search management platform to large search advertisers and agencies. He writes extensively on paid search on the ClickEquations Blog and Twitter @clickequations.

Connect with the author via: Email


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • PatrickSMC

    “The bid change not only affects our performance in auctions we were already entered into, but it also makes our keyword eligible for more or fewer auctions. But we have no way of knowing about our participation in these additional auctions – Google doesn’t offer an ‘auction count’ metric.”

    What about impression share reports? That’s been around for quite awhile.

  • dnrothwell

    Thanks for this excellent followup article.

    I agree the video is insightful and revealing.

    And I now (as of today) also believe the point about bid changes and appearing in other auctions and positions as a result, although I didn’t when I first saw the video. Seeing your post in my Google Alerts has prompted me to offer the following experience for consideration and comment by others.

    This afternoon I’ve just been doing some advertiser research, which involves typing a keyword, seeing what ads appear, and going through all the pages of search results carrying ads for this keyword (tents) – there were about 140 advertisers (UK).

    I noticed several things, some of which I’d seen before, but which now “seem” to gel and make a bit more sense against this backdrop.

    Firstly, to confirm I had actually reached the end of the advertiser results, I noticed the See your ad here » link which shows when all ads have finished, before starting over with the top adrank ad again (rather than More Sponsored Links >> which indicates additional ads to come on subsequent pages).

    When I was going through each page, sometimes I would see the same advertiser ad that I had seen on an earlier page, but now in a lower position – a different auction, presumably triggered both by me (since I refreshed the search by going to the next page), and by other users who are all searching just like me.

    Some advertiser ads also had disappeared from the auction when I had visited their website, and then gone back a page to the results page carrying their ad which I had previously seen and clicked through on.

    Perry Marshall’s observation of the “search and search again” phenomenon where ads quickly disappear and change position if you repeatedly refresh your browser may also be implicated in this auction discussion.

    What I think we need to consider are several points:

    1. Quality Score which governs the AdRank position is computed REALTIME for every search (i.e every auction) and is therefore potentially variable, because of broad keyword expansion, CTR, different ad text being split tested, restricted campaign delivery, ad optimisation and rotation, exhausted budgets, time of day, scheduling, advertiser competition etc.

    2. Since we’re talking realtime, you’ll only ever acquire data that pertains to a tiny snapshot of the whole auction activity over an instant of time – the next search that happens, or if you refresh your browser or go to the next page of results, and the auction has already changed.

    3. So the best data you can ever acquire is going to be a series of averages over a specific time period, and the longer the better to avoid glitches. So it’s overall trends you need to follow, rather than snapshots which could go up and down quite a bit.

    Maybe the only way to ensure maximum consistency and Impression Share (for an individual advertiser) would be to be in the happy position (with full comprehension of profitability of course) of being able to set a huge daily budget guaranteed to buy every click on the market (and then some), a high bid price to ensure consistently top position in the auction, accelerated delivery, for exact match only with a single ad.

    I wonder if anyone’s ever been able to do that? (I bet they have). I’d love to see those reports!

    Seems to me that sometimes we actually can expect “too much” transparency from Google, and that they have a highly automated machine which even *they* cannot fully explain all the results from, or offer reporting or tools which can help us dig deeper.

    They are still developing and refining it after all. I cite the Ads Diagnostic tool which is a great (fairly) recent functionality, but why should it be a painful, time-consuming and bug-ridden process which you can’t run as a scheduled report? I expect (or hope) that will eventually appear, as many things eventually do (remember when we could not even schedule our campaigns? I go back more than 5 years with all this…)

    So AdWords sometimes seems to me to be a bit more like Quantum Mechanics than we would actually prefer (by observing the experiment, you actually participate in it, and alter it without meaning to), and that sometimes we may need to go a little easier on our opinion of Google – after all, there’s no other system like it!

    Comments and feedback most welcome…

  • Craig Danuloff

    Thanks for sharing those ideas. I do think/assume that what happens to one person either looking at page after page or refreshing the same page may not be at all what would happen to different people seeing successive search results.

    Google ‘knows’ you’re reloading the page so it makes sense that they see if different ads catch your eye. The mechanics of how they treat page 2 onwards I don’t know either, but again it makes sense that the react to knowing you’re still flipping pages. I guess once I feel page 1 is under control I’ll start worrying about page 2….

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide