Blind Study Finds YP Local Results Beat Google, Bing, Yahoo

YP logoA “blind” study commissioned by directory publisher and local ad network YP, performed by Crowdflower, found that YP’s local search results were better overall than those of search engines. This was measured as a function of user satisfaction rating. Thousands of queries were compared over a year-long period.

I can hear the skepticism: “It’s sponsored study” and “How can they be better than Google’s local results?” Here’s what the study says about the methodology:

YP contracted CrowdFlower, an independent third-party, to employ an innovative and impartial methodology for the relevance measurement of local search results on YP and other sites. For over a year, CrowdFlower worked with YP to test approximately 13,000 local search query results per month to track user satisfaction (as a proxy for search relevance) of YP.com local search results and those of other popular local search engines.

The study matches YP local search technology against other popular local search engines: Bing local (bing.com/local), Google Maps (maps.google.com), and Yahoo! Local (local.yahoo.com). They were chosen for the study because of their high user reach and ability to offer search results across all local business categories. The search queries in the study span a wide range of local business categories. Sites that focus on a narrow set of vertical categories, such as restaurant or entertainment-heavy sites were not included because their scope does not match the breadth of this study.

In the evaluation task, search queries are presented to contributors who are asked to rate their satisfaction with the relevance of the results. To eliminate possible bias, the search results are presented in unbranded lists that show only basic listing information such as business name, address, and phone number.

YP study screen

I was told that this study was originally intended for the purpose of internal benchmarking. It wasn’t undertaken for marketing or PR purposes. Only in the end did the company authorize Crowdflower to release it.

As the discussion above mentions, users were not exposed to the brand of the search site. Above is an image of the type of screen study participants saw in every case, regardless of the source.

As the screen indicates, users in each case were asked to perform a query and then rate the result on a scale from “invalid search” to “perfect.” Each query had both a keyword and a location (city, state, zip). The distribution of query types is illustrated by the chart below:

YP query distribution

Crowdflower explains the query selection process further:

The majority of the queries (approximately 65%) are randomly selected from user searches on the YP.com site. The remaining queries are sampled from the most commonly searched categories (High Traffic), YP’s top monetized categories (Top Monetized), and a control set (Monitoring) that remains constant month over month.

If there’s any “bias” in the study it would be in the query selection process; this was not a randomly selected group of queries. However, that doesn’t account for the results.

Crowdflower reported that in November 2012, YP search results “were found more relevant compared to Bing local, Google Maps, and Yahoo! Local. Users were satisfied with YP.com search results 86% of the time, compared with 83% for Google Maps, 82% for Yahoo! Local, and 74% for Bing local.”

YP search satisfaction comparison

There are other charts like the above showing a comparison of user satisfaction and performance by business name query and business category. In each case YP delivered higher local search relevance/satisfaction overall than Google, Yahoo and Bing in that order. The following is a more detailed breakdown of categories where YP performed better and where its results were comparable to the competitors.

YP search category comparison

Microsoft often makes the claim that its results are better than Google’s but Google’s brand strength blinds people to that fact. People debate the accuracy of that claim. This may well be a comparable situation, where YP did do a better job of satisfying user expectations than its search engine competitors but Google’s brand makes that hard for people to accept.

I’m especially curious about what people think of these findings and whether you believe them or remain skeptical.

Related Topics: Channel: Local | Google: Maps & Local | Search Engines: Maps & Local Search Engines | Top News

Sponsored


About The Author: is a Contributing Editor at Search Engine Land. He writes a personal blog Screenwerk, about SoLoMo issues and connecting the dots between online and offline. He also posts at Internet2Go, which is focused on the mobile Internet. Follow him @gsterling.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • FrankReed

    Honestly, Greg, I am always skeptical :-).

    What YP can’t offer is what happens after the search and it can’t add value for a business like being in the Google ‘ecosystem’ can. And honestly, Google is where people search so businesses need to focus there. Sometimes people will need to do a search in Google just to GET to YP!

    Unless the Google Maps results are just pathetic, the ‘Google habit’ is hard to break for what appears to be a nominal ‘win’ for YP.

  • Greg Sterling

    Certainly people won’t change their behavior as a result of this survey. But it’s interesting to see the “cognitive dissonance” that it creates

  • http://twitter.com/zanblue Zanblue Info
  • http://steveplunkett.com @steveplunkett

    the sample set is those users ALREADY on YP.com – so search results WITHIN the website will be biased? no?

  • Greg Sterling

    That’s not correct. This was a crowdsourced research project using the Crowdflower distributed workforce, not a survey drawn from existing YP users.

  • http://www.facebook.com/jacobpuhl Douglas Jacob Puhl

    Like Frank, always skeptical as well, especially with ‘studies’ YP touches. After seeing so many yp-sponsored OUTRAGEOUSLY fallible, ill structured and bias studies about how people are still using the yellow pages, I have a hard time taking anything like this seriously. Then, there just the general sniff test/intuition that tells me the google search results are much much much better than the yp results (or else we’d all use yp.com, right?) In large part, the market kind of dictates what they find useful by actually using that product.

  • http://www.datadrivenseo.com/ Data Driven SEO

    Seems like a study designed to convince small business owners that YP is “better” than Google. Now their relentless sales staff has another tool in their biased tool belt to sell to unassuming small business owners that don’t understand online advertising. Hey Mr. Auto Repair Technician, did you see the recent study about how YP is better than Google, Bing, and Yahoo?

  • http://twitter.com/number1george George Freitag

    This is comparing YP results to Google Maps? If it compared it to Google.com or even the Places section it might be worth more. But no one searches like that in Maps.

  • DJuse2926

    I bought a brand new Fiat Panda since getting a check for $8686 this –
    four weeks past and just over ten/k last-month. with-out any question
    its the most-financially rewarding I’ve ever done. I started this 3
    months ago and immediately made myself over $70, per-hr. I went to this
    web-site,, Ask25.com 

  • Erin Rose

    This is an unbelievable farce: Crowdflower not even close to being an “independent third-party” because CF is THE vendor for YP for their outsourced data aggregation! So asking them to conduct this “analysis” is the same as any other company asking a vendor to rate their work for them. Calling it blind is truely shameful.

  • https://twitter.com/sanketpatel Sanket Patel

    I think is very hard to accept that search more accurate result from YP comparatively to other search engine. Users always get perfect local search result from Google, yahoo, bing.

  • http://www.facebook.com/saib.wafa Wafa Sayeb

    Certainly they wont as you said, but it interesting to know that some other engine beated Google !

  • http://www.ecape.com/ Julie Brooks

    Who cares? No one uses yp.com. They use Google. And the ones who use Bing are only using it because it’s the default on their browser. They barely know what a browser is to begin with, and they think they’re “Googling” something when they’re really using Bing. My local tavern has a hamburger that’s 500% better than McDonald’s ,but no one knows about my local tavern, they go to McDonald’s. YP is making the classic error or assuming consumers will choose quality over familiarity and habit, They won’t.

  • Lakisha Caudle

    If you think Nancy`s story is cool,, in the last month my friend’s brother worked and got paid $5193 grafting from there house and they’re neighbor’s mom`s neighbour has done this for 3 months and made more than $5193 part time from their pc. apply the information on this site, jump15.comCHECK IT OUT

  • polymath

    Local search is hard and all the major providers have problems. But could this type of study show who has the least problems? I don’t think so.

    Let’s say I was a contributor in the Crowdflower system, and I received the “pub in Houston, Texas” question above. How would I answer? I’ve never lived in Houston. Those look like they could be pubs, maybe I’ll say “perfect.” Or, hmm, one of them might not be, because it doesn’t have “pub” in its name. Maybe I’ll say “okay.” But regardless, I have no idea what the good pubs are in Houston. I don’t know whether the addresses or phone numbers are correct or whether they’re still open. That’s what I care about when I’m doing a local search, and as far as I can tell from the description you quoted, none of those things were tested!

    Where’s Foursquare on their comparison list? If I were to put my money on someone beating all these guys in a head-to-head comparison, it’d be them.

  • Guest

    Yp.com is a great source. Can’t take away from the power of Google, Bing and Yahoo. Our Agency works with both platform. It really boils down to the industry and as a marketing consultant I present the fact, research, traffic to my Client and recommend the best of the two or both if the budget permits.

  • Sal Torres

    Yp.com is a great source local source along with their network. Though, you can’t take away from the power of Google, Bing and Yahoo. I myself can’t agree 100% with this article. Our Agency works with both platforms. It really boils down to the industry the Business is in and as a marketing consultant I present the facts, research, traffic to my Client(s) and recommend the best platform. #localmarketing

  • http://www.facebook.com/leons.locmelis Leons Ločmelis

    YP is unusable for me. Searching Pizza near Rīga make “temporary glitch” in results :(

  • Ross Dunn

    YP is a joke. They are dinosaurs who need to realize their game is up. Over 20% of my new Canadian clients in 2012 came to me with their YP budgets because they were tired of the garbage service they got. YP is clearly trying desperately to cling to their old foothold as a monopoly. A “blind” report indeed.

  • Jason Chesnut

    The biggest mistake being made by both YP believers and Online Search Engine believers is not understanding that it all depends on the market the business is in. This includes both services provided and where they are located on a map. A smart business owner wont read this article and decide to invest all of his marketing dollars into YP programs. But he also shouldn’t invest all of his marketing dollars into Online Search Engines. A smart business owner will learn where his/her customers are, how and where they go to find him and then invest there. It is wrong being so black and white when it comes to marketing. This does the business owner a disservice and confuses the backbone of our economy, the small business. Consumers are a lazy bunch, and we will search for a business using the easiest and most convenient way possible. For millions of these consumers that is done by simply flipping open the print Yellow Pages, and for millions of others it is jumping on their tablet, laptop or smart phone to hit Google, yahoo, or Bing. Personally, I use Google to educate my self when searching for products, brands or information. But, I use YP Print and/or YP Online when I’m ready to purchase at a local location. Both YP and Online Search can be vital to the success of a business. The trick is, how much of each one will be the most effective.

  • Jason Chesnut

    They use Google Maps if they are looking for local business. Google has prioritized local search due to the thumping they took from competitive local search sources. Look up any business on Google and the first thing you see is Google Maps showing you the local businesses that matched your search request. Google Maps was created and prioritized by Google to compete with sources such as YP.com.

  • Jason Chesnut

    I feel bad for your clients. They deserve a better consultant that is not so one-sided, and that has the success of their clients business in mind rather than his own commissions.

  • Manny K

    Greg, the users are not, but the queries are from YP’s log. The ‘Top Monetized’, High-Trafficked segments – YP seems to have optimized to these handful high volume queries (who searches for ‘Security Control and Monitoring Systems’ otherwise?) They seem to be hardcoded category matches. Google/Yahoo are much better overall, because they also handle NPL and long-tail queries better.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide