Focus Group Study Offers Good & Bad News For Bing

TechCrunch yesterday reported on the results of a focus group conducted by NY design firm Catalyst Group comparing usability on Bing and Google. The study showed that in many instances Bing was preferred but that users already familiar and comfortable with Google would likely stay with Google. What’s partly interesting about this is that the results can be read either as confirming the challenge Microsoft faces in gaining adoption for Bing or to argue that for the first time Microsoft has a “real contender” on its hands and that Google should be concerned.

The focus group consisted of 12 people in New York who used Google as their main search engine. No one in the group had used Bing before. Users were asked to conduct a “hotel search” and a “shopping search for digital cameras” on both sites. According to the discussion in the slides, “users completed both searches on one site and then repeated the searches on the other site. The task order was rotated to mitigate bias effects.” The firm also collected eye tracking data. Users were interviewed about their experiences and there was also a short written survey they filled out. (One must be cautious about generalizing too much from a focus group, but the results should not be dismissed either.)

The following were the Catalyst findings:

  1. Most users preferred the Bing visual design and thought that it was slightly better in terms of organization and refinement options.
  2. However, despite some positive factors in Bing’s favor, users thought that both search engines produced equally relevant results. Overall, most users stated they would continue using Google over Bing and were unlikely to switch.
  3. For the camera search, Bing attracted 150% more user viewing time to the ad space at the top of the search result.

Here’s a slide that shows the survey scoring and user preferences:


Note that on “visual design,” “organization” and “refine & filter options” Bing is preferred (the dark boxes). Relevance is a tie for most people in the group. Also note that the “overall reasons for preferring” seem to contract the scoring in the individual categories. Google wins 8 to 4 based on:

  • Familiarity
  • Use of other Google apps (probably Gmail, Toolbar, Maps, Calendar)
  • Enhancements in Bing not enough to convert

Overall people like the design and interface features somewhat more with Bing.

In the run up to the Bing launch I had a couple of conversations with Microsoft in which I provided my response to the site and user experience. The things I liked about Bing are mirrored in the feedback provided by the focus group above. Microsoft echoed back to me that the design and UI scored well among testers. And “early returns” from comScore, Hitwise and others have indicated that Bing has gained some traction with the general population of searchers.

Let’s assume for argument that these focus group results are representative of a broader population of searchers and that the look and feel and certain features of Bing are indeed preferred. Let’s also assume that relevance is also basically perceived as a tie or that users cannot see significant differences between the engines. If you’re Google you can’t feel too good about any of that. It adds some weight to the oft-repeated yet otherwise empty remark “the competition is just a click away.” Yet if you’re Microsoft you see these generally positive results together with the “we’re unlikely to switch” statements and you’ve got to feel frustrated. But maybe you’re also encouraged.

In the context of this study, perhaps one of the most interesting findings is the reference in the Google preferences column to “already using other Google apps,” showing how Google’s larger “ecosystem” of tools, sites and apps emerges as a reinforcer of loyalty and continued usage. Related to that is the “inertia” surrounding familiarity with Google. Usage begets usage.

It would appear then that if Bing can gain some regular usage it will increase its own “familiarity” among searchers and a potentially build a cycle of increasing usage. Advertising and the several “default” search deals that Microsoft has done (HP, Dell, Lenovo, Verizon) would appear to have meaning then in exposing people to Bing and getting them to try and use it.

Again, I don’t believe that Bing is a threat to Google in any major way today. By the same token, without taking this too far, these focus group results suggest to me that despite points for “familiarity” there is some fatigue or even perhaps boredom with the Google UI.

If you want to take a look at the heatmaps and other specific comparisons, you can see and download the Catalyst Group slides here.

Related Topics: Channel: Other | Features: Analysis | Google: General | Microsoft: Bing | Top News


About The Author: is a Contributing Editor at Search Engine Land. He writes a personal blog Screenwerk, about SoLoMo issues and connecting the dots between online and offline. He also posts at Internet2Go, which is focused on the mobile Internet. Follow him @gsterling.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • jamesdmoran

    This was an interesting study, and I’m excited about any possibility that Bing forces Google to improve.

    I wonder whether we’ll see the view time of Bing’s top ads decrease over time, as more users grow familiar with the interface and become trained to ignore the sponsored listings.

  • danieltrrice

    While I’m interested in how Bing performs from an SEM standpoint, I find it hard to take a report seriously that only has 12 geographically isolated respondents. That’s just poor market research.

  • realtrack

    Hey very nice article

  • nickstamoulis

    Hi Greg, This is very interesting data, most of all that Bing “looks good” but does not have the type of relevant and on target search results that Google has…I personally think Bing is the last hope effort for Microsoft in search…

  • tag44

    Hey Greg, thanks for the post and for sharing the information with us.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide