More Ads = Better Ads = Better User Experience: Microsoft’s Success Formula?

In new numbers just coming out from AdGooroo, it looks like Microsoft is closing the gap on Yahoo in terms of advertiser share for the last quarter of the year, moving to a narrow three point difference (19.4% vs 16.4%) from a 17.6% gap in the third quarter. Also, in the report, it seems that Google was fairly aggressive in cranking up ad coverage while Yahoo stayed flat and Microsoft actually backed off on ad coverage, focusing on ad quality control and a more holistic search experience.  Finally, AdGooroo is forecasting the strongest quarter ever for Google and Microsoft, while showing Yahoo as fairly anemic, probably due to advertiser’s doubts about Yahoo’s future.

So, what does this have to do with user behavior? Well, it’s simple. As go the ads, so goes the user experience. And these shifts in the marketplace could be very indicative of correlated shifts in user experience. It seems like Microsoft is finally getting the message while, ironically, Google seems to be willing to experiment with one of the core tenets of their success.

The Motivation behind Microhoo

Of course, we’re freshly abuzz with Microhoo rumors and the AdGooroo report indicates that timing couldn’t be better.  They also pulled out a very telling Ballmer quote from December 5^th:

“We’re fully prepared to compete without any partnership with Yahoo. We don’t need to act. Would it be advantageous for both of us to make a deal? Look, the fundamental basis for doing the search deal with Yahoo has to do with critical mass in the advertising marketplace. It doesn’t have to do with technology, or any of these other things, it really is a market phenomenon. Together we would have more advertisers… which means we’d have more relevant ads on our page. We’d have higher monetization levels possible in front of us because there would be more people bidding on more key words. Most importantly, Google would have perhaps a real credible competitor sooner.”

Ballmer is talking about advertiser critical mass and Yahoo’s ability to contribute significantly to that critical mass seems to be evaporating quickly, if the AdGooroo numbers are any indication. If this is the sole motivation for Microsoft to do a deal, Yahoo’s marketability has a very narrow window indeed. I won’t talk about the value of Yahoo’s other assets or any of the nuances of a potential deal. That’s not the purpose of this column. But there’s another very interesting line in here, from the user perspective, and to understand the importance of it, I have to share the results of an astounding study we did at Enquiro for, you guessed it, Microsoft.

First, let’s read between the lines in Ballmer’s quote:

Together we would have more advertisers… which means we’d have more relevant ads on our page.

Ballmer goes on to talk about monetization, but what about perceived relevancy on the part of the user?  Some time ago in this column I talked about the importance of relevant ads, especially when they’re at the top of the Golden Triangle.  If poor ads showed at the top, we knew anecdotally that it impacted the user experience. We found empirical evidence of this in our second eye tracking study, when, in the midst of the study, Microsoft was switching from their Yahoo ad partnership to showing ads from their own inventory. The MSN (this was pre-Live) inventory was a fraction of Yahoo’s, which meant the relevancy of top of page ads took a major hit, right in the middle of our data collection.

When we started analyzing the data, we saw a significant change in user behavior over the MSN sessions. At first, we didn’t know what was going on. In a number of cases we saw virtually no interaction with top sponsored ads. It was almost as if banner blindness was occurring. We saw 86.7% of the sessions had their first fixation in these top sponsored ads, but only 55% stuck around to start scanning. This was a significantly higher drop off rate than we saw on Google or Yahoo.

Enquiro Eyetracking II Study, 2006

When we realized that in the affected sessions, the ads were coming from a different inventory, we divided the sessions into two groups for further analysis. When the ads came from Yahoo’s inventory, users spend an average of 4.93 seconds looking at them, which represented 41.6% of the total time on the page. These ads also captured 42.86% of the click throughs. But when the ads came from Microsoft’s inventory, users spent an average of 1.5 seconds on them, representing just 11.48% of the total time on the page. And these ads captured only 5.8% of the click throughs.

So, obviously poor ads meant much lower monetization on the search page, which is what Ballmer referred to. But this just meant users were skipping over the ads. How much of a negative impact did that have on the overall user experience? That was the question Microsoft asked us to prove in a subsequent study.

The study design was simple. We created a realistic search experience, then showed a mocked up results page and asked participants to interact with it as they would normally. Afterwards, we asked them four questions:

• Would they use the search engine to launch a similar query?
• Would you use the search engine for other queries?
• Would you recommend the search engine to a friend?
• Would you make the search engine your preferred search engine?

In the study, we divided the participants into two groups that were shown slight different versions of the search page. The only difference was the ad that showed in the top sponsored location. In the first group, the ad was a high quality, relevant ad. In the second group, the ad was less relevant (although not completely non-relevant) and of lower quality. That was the only difference. The other 16 results, including the other ads and algorithmic results, were exactly the same on the two versions. As I said, the results were amazing.

Enquiro & Microsoft, 2007

Just one ad, in a critical position, could substantially impact the user’s perception of the quality of the search experience. As I’ve said, over and over, in search, perceived relevancy is much more important than real relevancy, and with the current layout of the search page, a huge factor in perceived relevancy is whether the first thing you see, which is usually an ad, is a relevant match to your query.

If you take the results of both of these studies, it shows how important a big ad inventory can be for the overall user experience. The more ads you have to choose from, the more likely you can show a highly relevant one in that critical Golden Triangle real estate.

Microsoft’s on the right track

Finally, it appears the message is getting through at Microsoft. If the recent AdGooroo numbers are any indication, it appears that the importance of top of page relevance is being understood in Redmond. And, once it is, it creates a virtuous cycle that simultaneously improves user experience and monetization. This is something Google has always understood, but those same AdGooroo numbers point out a troubling trend in Mountain View which I hope is just a product of seasonality in advertising.

AdGooroo 2009

If top of the page relevancy is so vitally important, a search engine has to abide by this rule: If you don’t have a highly relevant ad, don’t show it at the top of the page. In the past, Google has always had a much firmer grasp on this rule than Microsoft or Yahoo.  But if we look at trends in ad coverage across the three from AdGooroo, it looks like the grasp may be tightening in Redmond (Microsoft) and slipping a bit in Mountain View (Google). Microsoft looks to be actively tightening up ad quality, resulting in fewer ads being shown, while Google seems to be experimenting with showing more ads. Yahoo continues to be the most aggressive in showing ads.

What isn’t shown in the above graph is how many of these ads show up in the critical top positions. I can say anecdotally that Google seemed to get more aggressive in this as well in the last quarter, while Microsoft seemed to be backing off. There are a number of complex and interrelated user behavior factors at play here that are beyond the bounds of this column, but showing ads at the top of the results is probably the single biggest critical factor in perceived relevance and, therefore, user experience. Someday, Ask and Yahoo may realize this, but I suspect it will be too late. I doubt they’ll ever make the correlation between this and their declining respective market shares.

As I’ve said, it’s not really surprising that Google’s ad coverage increased during the fourth quarter, given the retail importance of the quarter. It will be interesting to see if the increased aggressiveness persists through 2009.

Back to Ballmer’s critical mass

To sum up, relevant ads, shown appropriately, are a critical user experience factor. And the bigger the inventory, the greater your ability to consistently deliver relevant ads. If you take Ballmer at his word, that’s the only reason Microsoft wants Yahoo. Let’s again use the AdGooroo numbers to see what this window of opportunity looks like.

AdGooroo 2009

It appears that Yahoo is trying to sell cruise packages on the Titanic. Obviously, both Google and Microsoft are picking up the advertisers that are jumping overboard.  The question is, can Yahoo bail out before it sinks forever?

For more on the latest AdGooroo numbers, go to their press page.