Varian Shares AdWords Secrets – Will He Bring Transparency To Google?

I’m a big fan of Google’s Chief Economist, Hal Varian. When Dr. Varian speaks we gain authoritative and important new information about how the Google AdWords auction really works. Information that we had no way of gaining on our own but that significantly improves our ability to manage our accounts and campaigns.

I also like him because there’s a Joe Biden-like quality to the way Dr. Varian delivers the truth (and I mean that as a genuine compliment.) His communications are based first in facts and only secondarily packaged and limited to fit the party line. The result is that along the way we learn, or have confirmed, a truth or perspective that the PR flacks and marketing spinners would probably rather have remained unspoken.

Dr. Varian’s first major communication was the ‘How AdWords Auction Works’ YouTube video. It’s the “Subterranean Homesick Blues” of AdWords videos—a long complex stream of previously unknown truths which demands repeated viewings.

[youtube]https://www.youtube.com/watch?v=K7l0a2PVhPQ[/youtube]

In it, Dr. Varian explained the math behind the calculation of both position and CPC in AdWords. Suddenly we could quantify the economic cost or benefit of Quality Scores and Quality Score changes.

As Paris Hilton would say if she ever spoke without irony, “that’s huge.”

Now in a post on the AdWords Blog Dr. Varian and his team report that they have verified that conversion rate doesn’t vary much based on ad position. This is critical information in our quest to intelligently set bids and build bidding strategies. Calculating the “right” bid without knowing that conversion rate was relatively constant was difficult, to put it mildly.

If conversion rate is relatively constant, then revenue per click is relatively constant (we’ll assume for the moment that basket size isn’t varying by position either). This means that if we can determine the variation in cost-per-click at different positions we can rather easily decide the “right” bid to maximize revenue or profit.

All of this means that Dr. Varian eviscerated one of the core assumptions about PPC bidding—that conversion rate varies dramatically by position—which was held by a lot of smart people and around which many an algorithm has been written.

To be fair, there are more than a few people who remain skeptical of Dr. Varian’s claim. I’ve seen blog posts and spoken to some smart and experienced paid search practitioners who believe they’ve seen data which contradicts his point.

Is Dr. Varian wrong? Are his claims too general? Are there marketplaces or situations in which conversion rate isn’t linear? Or was he able to hold the variables more constant, in terms of statistical analysis, than mere mortal paid search advertisers can because he has access to more and purer data? We don’t really know, but until proven otherwise I’m giving Dr. Varian the benefit of the doubt.

A new twist

But while the first impression is that Google’s release of this statistical analysis makes bidding easier, an aside Dr. Varian made as part of his explanation muddies the water again.

It’s in the comment where he explains that sometimes when you raise your bid your average position goes down. He says that as your bid increases the keyword you’re bidding on becomes eligible for new auctions. And in these new auctions, which you’ve just barely bid high enough to earn an ad rank that made you eligible, you’re likely to appear in lower positions, thereby lowering your overall average.

In other words, when you increase your bid, your ads should generally rank higher within the auctions where they were already showing—this is probably your intention when you raised your bid. But at the same time, a possibly positive but probably unintended consequence occurs: those same keywords with their new higher bids now trigger impressions for queries that they failed to trigger before. And for these new queries your ads are generally appearing in lower positions.

All of this is perfectly reasonable. The fact that the higher bid expands the range of search query eligibility is in line with what we know about the auction process, even if it’s a result many of us hadn’t considered. The problem is that it’s another example of AdWords giving us data that conceals what is actually happening. If you just looked at the data and made a reasoned analysis of it you would very likely draw the wrong conclusion. With a full understanding of the situation you’d know you really can’t draw any conclusion at all.

We’re buying advertising, getting data back which is ostensibly provided so that we can learn from our results and make rational decisions about improving our campaigns. And yet it turns out the data is inadequate and inconclusive. This isn’t the only situation where this is true, but it is a great example of how AdWords gives the impression of transparency while failing to provide the data we need to make smart optimization decisions.

Here’s what we now know:

• We bid on keywords and review reports showing our clicks, cost, revenue, and position.
• Along the way, we may raise or lower a bid to see the impact of that change on our results.
• The bid change not only affects our performance in auctions we were already entered into, but it also makes our keyword eligible for more or fewer auctions. But we have no way of knowing about our participation in these additional auctions – Google doesn’t offer an ‘auction count’ metric.
• The average position metric isn’t the indicator of impact we’d normally expect. When average position goes up, down, or stays the same after a bid change we have no way to correlate the two events. A basic assumption most of us had about bidding and reporting is shattered.
• For the same reason, we can’t draw any particular conclusions about resulting changes in our click volume or average CPC. Our bid change might be having the exact result we wanted – rising positions, more clicks, higher CPCs, and increased net revenue – when run against the same queries we were always matching, while at the same time exposing us to a whole new set of queries which turn out to be poorly converting wastes of money that lower our numbers to the point where it may be impossible to tell that our bid change largely had the desired effect but unfortunately came bundled with a horrible side effect.

Of course, we can simply look at the net result, conclude that our bid change was a bad idea, and reverse it.

But that’s not the way many of us believe we should have to manage our keywords or accounts. We’d like data to be provided at a high enough resolution so that we can make precise and deliberate decisions. If there are multiple auctions going on in relation to a single keyword, and we can make money in some but lose money in others, we’d like the data and options that would allow us to choose where and how we participate.

The fog is lifting

AdWords advertisers have spent a lot of time and money feeling around in the dark. The information Dr. Varian shared in his video and his blog post are very helpful points of light. The new ‘Bid Simulator’ in the new AdWords interface is another.

It’s great to see Google sharing this information, and providing these tools. I look forward to this trend continuing to the point where paid search advertising on AdWords is truly a fair and transparent—although still competitive—marketplace. I think that’s where Hal Varian is taking us.