Time To End The Bull Search Engine Share Figures?
Yahoo’s up again in comScore’s latest search rankings. Reason? It seems largely due to non-search “slideshow” activity. Solution? Time to end the bullshit way search share figures are calculated. Chance of success? Probably nil, but it’s worth hoping for. Today’s figures, for share of searches in the US handled by the major search engines, show […]
Yahoo’s up again in comScore’s latest search rankings. Reason? It seems largely due to non-search “slideshow” activity. Solution? Time to end the bullshit way search share figures are calculated. Chance of success? Probably nil, but it’s worth hoping for.
Today’s figures, for share of searches in the US handled by the major search engines, show Yahoo has continued to increase its share of searches for the third month in a row. The trend:
- February 2010: 16.8
- March 2010: 16.9% (up 0.1%)
- April 2010: 17.7% (up 0.8%)
- May 2010: 18.3% (up 0.6%)
How Yahoo Games Search Share
Prior to March 2010, Yahoo had seen a year of decline. What happened? One big change was that Yahoo started ramping up “slideshows” at Yahoo News to manufacture more searches. For example, if you’re on the Yahoo News home page, you’ll see a “Most Viewed Photos” section like this:
If you click on a photo, you get tossed into “slideshow” mode:
As you browse through the pictures, each click generates a new “search,” as you can see below:
In the example above, the person viewing the photo probably doesn’t realize they issued a query for “Police stand outside police headquarters.” They did.
Below the photo, they got news search results that don’t match the query they issued. But that’s OK. They probably didn’t notice the results were even there. In addition, they also got ads, none of which are targeted to the search query. Advertisers, take note.
Each click to get a new picture changes the URL to a new search query, thus making it seem when the person did a new search — even though they really didn’t.
Sure, way down at the bottom, the search results changed. Chances are, the user didn’t even look at these. Almost certainly they didn’t perceive what they did as a search. But it got counted that way, and Yahoo’s share has risen because of this.
Yahoo also says that it has ramped up the use of “trend” links like these on its home page:
Click on a trend link, and you get search results for that trend. Sprinkle links like these around Yahoo, and people apparently click a lot, Yahoo says, meaning more searches for them. Unlike with slideshows, at least these are more arguably search-like activities.
The Gaming Has Grown Searches
Let’s be clear. The two things above are the biggest drivers of Yahoo’s success. When I spoke with Shashi Seth, senior vice president of Yahoo search products yesterday, he made that clear.
Seth did argue that Yahoo has also been spending on radio ads to promote search since February. He sees those as “additive” to Yahoo’s recent successes.
I completely disagree. When search engines do ad campaigns, you usually see an immediate spike (if the ads work). More than a month after the ads started, the March 2010 figures showed a 0.1% increase in search share. Tiny. But in late March, Yahoo ramped up its use of slideshows. The following month, in April, it registered a 0.8% share. That’s a spike. That’s a spike related to slide shows and other activities to generate more searches on Yahoo itself, not by bringing in new people through radio ads.
Yahoo: We’re Just Doing What Microsoft Does
Counting slideshow activity as “searches” is bullshit in my books. Yahoo, oddly, kind of agrees. But Seth argues they weren’t the first to start the gaming. Blame Microsoft, Yahoo says. Yahoo’s just copying what they’ve been doing.
For example, here’s what you get at Bing Travel, an enticement to view slideshows:
Click on one of those, and yes, the same bullshit at Yahoo happens at Bing:
The picture you’re viewing is delivered complete with search results, even though you didn’t intend to conduct a search. Click through the slideshow, and you generate more searches for Bing.
Time For Game Reform?
What’s the solution to this gaming? You probably didn’t notice that both Yahoo and comScore put out press releases yesterday touching on the issues. This was damage control, anticipating that there would be a lot of complaining about Yahoo gaming the numbers when May 2010 figures came out today, as there was. The same thing happened when the April 2010 figures showed an increase.
The industry as a whole is changing rapidly, and as search continues to evolve, so will the measurements used to understand the market. People no longer search to find a list of blue links; they search to find answers in the shortest amount of time possible. We believe that surfacing the right information at the right time is more important than the number of total results delivered or number of traditional queries conducted.
As we push forward with new search features to bring relevant search results to more people in more places, we look forward to talking with folks like comScore and the industry at large about how to measure the new paradigm of search. Our goal is to help evolve measurement standards, definitions and metrics so the industry has what it needs to accurately understand trends in search share across different types of searches and different companies.
In July 2007, comScore revamped search measurement with the introduction of qSearch 2.0, which expanded the view of search beyond the traditional search sections to include searches at retailers, travel sites, directories, etc… anywhere that a user might submit a search. qSearch 2.0 captured the broad adoption of search technology throughout the web to improve the user experience.
Since that time, we’ve seen a wave of change across the web that has changed the very nature of a web page, from an object that is requested and delivered, to one that is a live platform that can integrate content from many sources. And search has changed along with it….
That said, the continued evolution of search and emerging innovations in how it is used to enhance user experience, calls for a thoughtful review of how we classify various types of searches, count them and report them. We want to ensure that we provide comprehensive and flexible measurement that meets the needs of the various constituencies in the digital marketplace. As our thinking evolves, we will include relevant stakeholders in the discussion and clearly communicate our thinking and rationale to the marketplace. While we will maintain the current method through the end of the second quarter to avoid reporting disruptions, we will aim to implement proposed revisions in the third quarter, ideally starting with the release of July data in the first half of August. Stay tuned to the comScore blog to find out more in the coming weeks.
They’re right. We do need better ways to measure search activity. But they’re wrong that this is somehow some new issue that’s come up.
Bad Metrics Aren’t New
I’ve been following search figures for a very, very long time — since 1996. Over the years, I’ve seen a shift from measuring search by the number of people who visited a search engine at least once per month (a laughable metric, but that’s how it was done) to the number of actual searches that were conducted. I’ve seen search engines not get counted because they weren’t considered search engines then spring onto the radar screen after a “reclassification.”
In addition, while search is evolving, we’ve long had search engines providing direct answers or “contextual” responses in the ways that the posts above would have you think are suddenly new. They aren’t. Search engines have used a variety of methods to try and “recirculate” people into doing more searches with things like related searches or answer box displays. What’s new is that they’re proving more effective gaming mechanisms than in the past.
Prior to this, the last big gaming success what the actual honest-to-goodness search games that Microsoft offers through Club Bing. Our SearchPerks – Microsoft’s New Prizes For Searches Program article gets into more detail about that, and the debate it raised.
Why No Prior Call For Reform?
Here we are more than three years after Microsoft actively used search gaming to generate new traffic, and we’ve not seen comScore nor any of the other major ratings services call for revisions on counting (or not counting) that activity. Why not?
The answer’s easy. Because it didn’t help in the long-term. Microsoft kept losing share, so it wasn’t worth trying to figure out whether games were helping them or not. They were clearly losing regardless.
Now Yahoo’s having success — and suggesting that Bing’s success is less to do all its marketing and more to do with slideshows. Cue the cry for reform again. But will it happen?
It should. Back in 2008, I started talking with some of the metrics companies about other ways that search could be measured. One thought is the number of search “sessions.” Not number of searches but a longer period of time when someone is exhibiting search activity. Go to Google four times per day for different things? Each visit probably generates several searches. Rather than count the number of searches, count the sessions.
That’s one suggestion. It’s not perfect, and we likely need a basket of metrics to fully appreciate what’s happening.
Will It Happen?
I hope we do see reform. I’ll push for it further myself, in follow up conversations with the various search engines and metrics companies over the company months. I’d like to see the big three come together on the issue. I’m just not that hopeful. Google, in particular, has little incentive to get involved.
Let’s be clear. With all the attention on Google from an anti-trust perspective, the company has a strong incentive to make it seems like it’s losing a bit in the search space. That’s especially so when from a search volume perspective, it’s doing just fine.
Consider today’s figures, which report search share in the United States for May 2010
- Google: 63.7% (down 0.7% from April 2010)
- Yahoo: 18.3% (up 0.6%)
- Microsoft: 12.1% (up 0.4%)
Google’s in trouble! But then again, look at the number of searches generated:
- Google: 10.158 billion (up 152 million)
- Yahoo: 2.908 billion (up 156 million)
- Bing: (up 101 million)
Google’s actually even with Yahoo on growth of actual searches and ahead of Bing, despite having a “share” that drops. It sounds weaker despite being strong.
Moreover, if the gaming keeps working for Yahoo and Bing (and gaming seldom has, in the long term), Google could crank up its own and crush the competition. It has so much search traffic that even the smallest changes might skyrocket the searches it handles. Perhaps it already has.
See my article from last month, When Losers Are Winners: How Google Can “Lose” Search Share & Yet Still Stomp Yahoo, for more perspective on search volume versus search share figures plus a chart with share and volume figures over time. I’ll update that chart into this article in the near future.
Postscript: Citi Investment Research sent this in an investor note:
When we exclude the incremental contextual/slideshow searches from YHOO and MSFT, we get a very different picture of query results.
Google’s share actually increases from 66.1% in April to 66.4% in May, up 30 bps M/M and up 140bps Y/Y. We note that this is a record share for Google.
Yahoo!’s share falls from 16.9% in April to 16.6% in May, down 30 bps M/M and down over 340 bps Y/Y.
Microsoft’s share remains constant at 10.8% in April and May, up 280 bps Y/Y, and is the lowest share since Nov-09.
It’s unclear to me where Citi is getting these figures; I suspect they’ve asked comScore for a special run. Over at eWeek, there are similar figures even though these aren’t listed in the comScore press release. I’ll be checking on this further.