Last week, I wrote The Lies Of Top Search Terms Of The Year covering how the top terms of 2006 from Google turned out not to be the top terms at all in raw popularity. I caught up with Google at the end of that week to talk about this more but didn’t have a chance to post a follow-up before Christmas hit. Now Google’s done a blog post – How we came up with year-end Zeitgeist data — explaining that the top searches list really should have been called the "top gaining searches" list. More on that below, plus some expanded comments from my talk with them last week and a top gainers list for the United States in 2006.
Let’s start with the key explanation from the blog post:
We looked for those searches that were very popular in 2006 but were not as popular in 2005 — the explosive queries, the topics that everyone obsessed over. To come up with this list, we looked at several thousand of 2006′s most popular searches, and ranked them based on how much their popularity increased compared to 2005. ("Bebo", for example, had very little traffic in 2005.) We also gave a bit higher score to searches with more traffic. Similarly, our "what is" and "who is" lists are not necessarily the absolute most frequent searches, but rather those that best represent the passing year.
As I wrote before, showing top "gainer" terms like this had long been a hallmark of the monthly and weekly top term "Zeitgeist" list that Google has released. The main problem with this end-of-year release was simply that Google didn’t explain it well. Here’s what the text on the 2006 top terms list used to say:
A year’s worth of search speaks to our collective consciousness, and 2006 is no exception. To compile these year-end lists and graphs, we reviewed a variety of the most popular search terms that people typed into Google. Click on all the tabs to learn something new — or confirm that you too reflect the spirit of the times. Except where noted, all of these search terms are global for Google.com.
And here’s what it says now:
A year’s worth of search speaks to our collective consciousness, and 2006 is no exception. To compile these year-end lists and graphs, we compared frequent queries this year against 2005 to see what sorts of things were top of mind. Click on all the tabs to learn something new — or confirm that you too reflect the spirit of the times. Except where noted, all of these search terms are global for Google.com.
I’ve highlighted the changed sections in bold. We go from having a list of the "most popular search terms" to comparing top queries in 2005 to 2006 and showing the gainers.
That’s fine. I understand the news value in talking about the gaining queries. The explanation should have been better, but now it is fixed, and it makes a lot more sense why something like "bebo" managed to top the 2006 list.
Now let’s dive a bit deeper into some of those terms. Recall my chart from Google Trends of the top terms on the list:
Bebo hardly looks popular in comparison to some of the others. But if you isolate Bebo, as you can see here on Google Trends, it has a much sharper rise from nothing to popularity.
What about MySpace? It was already popular in 2005 (in fact, the top gaining term), so how did it show up again in 2006. It kept gaining that much from 2005 levels.
Queries had to show gain, but it wasn’t as simple as taking the raw "top gaining" queries. Consider a search that happened only once in 2005 and then happens 10 times in 2006. That’s a 1,000 percent rise (if I’ve done my math right). If it happens 100 times, that’s a 10,000 percent rise. Small volume terms like this could have raw percentages that outweigh large volume terms like Bebo. That’s why volume was considered in creating the list.
"We didn’t trim that list. We trimmed low volume queries and a lot of stuff about us, because it seemed weird to put ourselves on our own list," said Google vice president of engineering Douglas Merrill, when we talked last week. "We didn’t annualize the numbers. It’s the slope of the growth line across a reasonable number of queries."
Explaining further, he said:
"With Steve Irwin [on the US list below], the spike is short but very sharp. You don’t have to be growing at the same rate all year long. You have to have a very high slope of that line and query volume to be interesting."
Now what about that stuff of Google that was dropped out? Again, I covered some of this in my previous article, how you can turn to something like Google Trends and see that queries for "google" on Google easily outdistance the overall volume of other terms. Even Merrill acknowledges this:
"We see a lot of people googling for google, which is pretty ironic," he said.
But that trend line of queries on "google" doesn’t rise, so even without the filtering of Google’s own brands, "google" as a top term probably wouldn’t have made it. The same is true for those adult terms that are removed. They’re popular, but they haven’t risen in popularity, so don’t make the cut.
How about YouTube? That’s a huge success story of 2006. Is it not on the list because it’s a Google property now? No. It just didn’t have the right combination to make the overall worldwide list. However, it did make the US top terms list that Google provided me, below:
4. steve irwin
9. zac efron
10. greys anatomy
As a result the list that they supplied is pretty much meaningless, and who is to say that the version released in 2005 is actually any better. So thanks for that Google – I’ll be sure to use Yahoo or Ask next time around; I might be able to trust them a little more.
The thing is, by rights I should have made more time and done the same interviews with all the other search engines about the lists they put out. That’s because to me, they are no more honest than Google’s list, at least if you consider that list dishonest or meaningless. Let’s go back with them again, and I’ll call out the single most popular term each has reported:
- Top Yahoo Searches
Of 2006 – From the "Top 10 Overall Searches" category (and that’s all the
explanation you get about this), the term is Britney Spears. I don’t believe
that. I think queries like "google" and "yahoo" and "porn" easily outweigh
those of Britney. If Google’s list is meaningless, as some are calling it, so
are these other lists. That’s why my original article was called "The Lies Of Top
Search Terms Of The Year" not "The Lies Of Google’s Top Search Terms Of The
Top Live Searches Of 2006 – Ronaldinho was the "Top Overall Search."
Right. Again, things like "google" and "sex" probably outstripped even the
great footballer. Note that Live Search also did a top gainers list and there
we see Bebo at the top, just like with Google.
- Ask.com Top Searches Of
2006 – I could perhaps see MySpace coming in as the top search of 2006 as
listed, but dictionary as the number two top term? I think not. More filtering
is clearly going on.
- AOL’s Top Searches Of 2006 – "Weather" was the top overall search. Not. I have no doubt there are a lot of weather related queries, such as "weather san francisco" and "weather chicago." But adding those all together gives you a top search topic, which is a different thing that a top search term. In addition, you can bet the usual "boring" queries like "hotmail" and "google" and "sex" have been filtered out.
Since we’re coming up to New Year’s, here are some resolutions all the search engines can consider for when they do the top searches of 2007:
- Publish an unfiltered list. I know you don’t want to. I know the
unfiltered stuff looks bad. But give us that top ten or top 25 anyway so we
can judge raw data on our own. Besides, don’t we have a right to really see
how we are searching, if you’re going to shove lists that tells us based on
your own perceptions?
- Explain everything. If you filter or skew, explain somewhere — but in an
easy to find location — exactly what you’ve done.
- If you are showing actual search terms, then call the lists "search term lists." If these are search topics — many different terms combined into one – make that clear and use a different label.