uTest’s “Battle of the Search Engines” Tests Bugs

Sep 15, 2009 at 9:31am ET by Barry Schwartz

uTest released a study they named “Battle of the Search Engines” which measured the relevancy of Google, Yahoo, and Bing based on “search results accuracy, page load speed, real-time relevance and usability.” The report found that there were over 600 “bugs” found in the test. By “bug” I believe they mean links that lead to dead ends, i.e. broken links. I am not exactly sure how that measures “relevancy” of the search result, but it does tell us some things.

In any event, here are the key points from the survey:

71% of respondents stated search results accuracy as the most important when choosing their search engine.
In overall accuracy, Google led the way with a top two box score (those rated as “excellent” or “good”) of 90%. Caffeine followed with 83%. Yahoo & Bing trailed with 53 and 42%, respectively.
Bing had the most reported bugs; however, it scored very high for usability and pics/vids search
More than 30% noted that Bing surprised them favorably; and 10% said that after testing all three engines, they would make Bing their default engine.

Here is the break down by search engine:

Google

130 bugs reported
8 percent classified as “showstoppers”
28 percent were Technical, 46 percent were Functional, and 26 percent were GUI

Bing

321 bugs reported
14 percent classified as “showstoppers”
29 percent were Technical, 46 percent were Functional, and 25 percent were GUI

Yahoo

70 bugs reported
10 percent classified as “showstoppers” and 50 percent as “high severity”
28 percent were Technical, 57 percent were Functional, and 15 percent were GUI

For the full report, click here do download the PDF.

Barry Schwartz is Search Engine Land's News Editor and owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry's personal blog is named Cartoon Barry and he can be followed on Twitter here.

See more articles by Barry Schwartz >

Share, Bookmark & Discuss This Article
More:

See more stories like this in the Members Library! Check out the Stats: Relevancy sections of the Members Library where this story is filed. Members also get access to exclusive video content, a members-only weekly & monthly newsletter, plus more. Check out all the benefits!