Stop Presses! Google Bested In Search Shoot-Out!

PC World has released the results of its most recent search engine shoot-out, and surprise, surprise, found that “Google is indeed the best search engine.”

However…

This wasn’t just a casual test, and Google wasn’t the best across the board with all types of queries and content. The author tested 55 services in six categories: general (text info), video, mobile/local, news, images, and blogs. The goal was to simulate how “real people” searched, so no special commands or modifiers were used. In the text category, the real surprise was in the two search engines that bested Google:

AlltheWeb and AltaVista, two Yahoo-owned services that are powered by the same Yahoo index. Here’s a chart summarizing the results of the tests:

search-shootout%20407.jpg

You can also see the full chart with comments online.

As these types of things go, this shoot-out was fairly well done, avoided some of the incorrect assumptions and conclusions that others have fallen into, and generally offers a pretty good snapshot of the state of search today. Of course, all testing of this type is highly subjective and can’t be used to draw absolute conclusions about the quality (or lack thereof) of any search engine tested.

Years ago, when Danny and I conducted a similar test we called “the perfect page test” several of the search engines protested that our conclusions weren’t fair. AltaVista’s then chief scientist Jan Pedersen wrote:

“Correctly assessing the relevance of an Internet search engine is methodologically tricky. All too often people resort to anecdotal tests based on a small number of favored queries — for example, someone’s name. This can be misleading, since no one query is a good predictor of overall relevance and small samples produce unstable and unreliable results. In contrast, truly informative tests, such as those sanctioned by the Information Retrieval community, and such as the one we run internally at AltaVista, average over a large number of diverse queries, typically several hundreds, to produce statistically significant results. We use a random sample of queries to avoid the judgment bias that can creep into a test using a ‘representative’ query set.”

PC World’s test was in fact the type of anecdotal test Pedersen describes, and so some of the biases he mentioned do surface in the test results. But the article does serve as a good reminder that there are other services out there besides Google that are in fact as good or better when it comes to searching various types of content. It’s a good read and worth the time to read. Read on in PC World’s Search Engine Shoot-Out.

Related Topics: Channel: Strategy | Stats: Relevancy

Sponsored


About The Author: (@CJSherman) is a Founding Editor of SearchEngineLand.com and President of Searchwise LLC, a Boulder Colorado based Web consulting firm. He also programs and co-chairs the Search Marketing Expo - SMX conference series.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://www.mattcutts.com/blog/ Matt Cutts

    The mainstream press may like to talk more about new Google products or features, but the search quality team at Google is quite focused and works very hard. It’s kinda funny; I went to mail the article to a few other Googlers to say “Cool, now how can we improve search quality even more for these types of queries” and another Googler had already beaten me to it. :)

  • http://www.searchlounge.org chrisf

    Wow, am I missing something? The PC World article barely describes the methodology used, though I think (can this really be true?) that they used 10 queries? They used 10 queries to evaluate engines that handle millions of real-world queries. Please tell me I’m misreading something…

    The article states: “Google is indeed the best search engine, even though two other services topped it–barely–in our text-search tests.” Ahm, OK…That’s confusing.

    And the notes in the chart are mostly about UI.

    I know that every relevancy test has its naysayers and normally I would have let this one pass like others I’ve seen, but I couldn’t because the chart takes it beyond anecdotal.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide