89% Find Search Engines Do Good Job Finding Information, But “Noise” Is Issue

Has Google’s relevancy gotten worse? A recent opinion poll suggests not, while at the same time confirming a concern that’s been rising in anecdotal accounts — there’s too much “noise” surrounding the “signal.”

Rasmussen Reports surveyed 740 adult Americans on January 4-5 about a variety of search engine related issues. The key question that caught my eye?

“In terms of finding what information you need, how do you rate today’s Internet search engines like Google, Yahoo and Bing …excellent, good, fair or poor?”

Most Rate Search Engines Well

In total, 89% found that search engines do a good or excellent job in finding information. Here’s the full breakdown:

  • 47% – Excellent
  • 42% – Good
  • 10% – Fair
  • 0% – Poor (technically between 0% and 1%, but specific figure not given)
  • 1% – Not sure

Does that mean Google itself is gaining such high marks? Maybe these are all Bing users? Unlikely. The survey didn’t ask which search engine people used, which was unfortunate. It did ask if people used more than one search engine at the same time. Few did:

“Do you generally use the same Internet search engine all the time?”

  • 78% – Yes
  • 19% – No
  • 3% – Not sure

Since Google is by far the most popular search engine in the US, it’s reasonable to assume that the overall satisfaction numbers indicated overall satisfaction with Google.

But Noise Is An Issue

If search engines are doing such a great job in general, and Google in particular, why have we seen a spate of posts recently suggesting that Google’s gotten worse? I think the answer is in another question from the poll:

“Which is a bigger problem when you use an Internet search engine – that you can’t find what you need or that your query generates too much irrelevant data?”

  • 70% – That your query generates too much irrelevant data
  • 13% – That you can’t find what you need
  • 18% – Not sure

Only 13% say they can’t find what they’re looking for. The answers are there, the “signal” that people want to tune into. They’re just surrounded by a lot of noise.

Another Poll With Seemingly Conflicting Findings

I think you see a similar frustration in a poll that Lifehacker just ran. This gathered nearly 10,000 responses to the question:

“Have Google’s Search Results Become Less Useful To You?”

  • 43.8% – Kind of/sort of, but it’s still the best way to get at the good stuff
  • 33.8% – Absolutely. The spammers have gained a significant foothold
  • 11.2% – I haven’t really noticed a change
  • 7.1% – I’d say no, or not to the point where it matters, at least
  • 3.6% – No, and actually, my results have been better and more convenient lately
  • 0.6% – Other

The headline on Lifehacker’s poll results story was “Over 77 Percent of Lifehacker Readers Say Google’s Search Results are Less Useful Lately,” which combined the two most popular responses, one that is totally negative and one that can be read either way (results are less useful, but Google’s still the best way to find things).

However, “less useful” doesn’t mean “useless” or that people aren’t continuing to find things with Google on a regular basis. To better illustrate this, you could also do a headline from the same poll saying that 2/3rd agree that Google results are still useful. Consider the side-by-side charts below:

The first chart combines the negative and mixed responses into the “Yes” slice. The second chart combines the positive, neutral and mixed responses into the “Yes” slice. Same poll, seemingly different conclusions — unless you understand that growing noise (and frustration with it) doesn’t mean that Google has suddenly become useless.

Good Remains Good Enough

Last year, I did a long look at Google’s results in my story, How The “Focus On First” Helps Hide Google’s Relevancy Problems. The point of that was to illustrate what the poll numbers above are showing: good is good enough for Google to win. As I wrote:

In a few minutes, give me a query, and I can usually find at least one result that doesn’t match the quality you’d expect to be in the first page of results on Google. If it’s an area I’m an expert it, I can do it even faster — and find more outliers. And if you go to the second page of results, it can sometimes be laughable. Google survives this because for the most part, a few good answers are good enough.

If Google really were as bad as some anecdotal accounts suggest, you’d see a loss of users, in my opinion. That’s based on watching the search engine space for 15 years now, observing what’s helped players rise and fall. Instead, Google continues to remain well above its closest competitor. One measuring company, comScore, just reported Google having record usage in the US.

But Google Comes Under More Scrutiny Now

By the way, Bing and Blekko also get by on “good is good enough.” As with Google, I can easily find irrelevant results on them without breaking a sweat. But they aren’t coming under the fire that Google’s starting to take because, in my view, they’re enjoying the underdog benefits that previously Google enjoyed.

As I wrote in my Blekko Launches Spam Clock To Keep Pressure On Google story last week:

There was a press love affair when Google first came out. There continues to be a consumer love affair, in my mind, that the Google brand on search results can make them seem better. There have been several studies in the past where just putting the Google logo on someone else’s results will make a consumer think the results are superior.

I think we’re finally seeing this slip back on Google. Just as its achievements were inflated into super-greatness, now its results are blown-up into huge failures.

You can see this in Paul Kedrosky’s piece from this week, when he writes:

We could get better algorithms, which is happening to some degree, with search engines like Blekko and others

I like Paul, a lot. I like Blekko and the folks over there, a lot. But make no mistake. Blekko absolutely does not have a better search algorithm than Google. It has a different search algorithm that is used against a completely different collection of documents than Google — and one that is probably only a couple billion pages in total (if that) versus Google dealing with tens of billions of pages.

No one knows who has the better search algorithm. Blekko, for all the attention it has gained, still has tons of learning to do — and the folks at Blekko have said as much, even as they ride the latest “Google sucks” wave.

Perception Isn’t Reality

Personally, I’ve felt that Google’s search quality hasn’t been as good as in the past. But my gut feel might be wrong. I remember the negative experiences far more than I recall the many times that Google works extremely well for me. I also use Google far more than I use Bing or Blekko — which also means I’m less likely to notice things that go wrong at those search engines. I also search for things I never would have tried in the past. My expectations have grown, over time.

I don’t think I’m unusual. For those who’ve written recently of Google’s “decline” in search quality, I think they’re all regular Google users and mainly writing from their guts. None of the posts I’ve seen appear to have done any robust testing of queries on Google and compared those to Bing, much less tried to measure changes in relevancy at either search engine over time.

Testing Needed, But Testing Is Tough

I’ve written several times recently — as I’ve done years ago — that what we really need are batteries of tests runs against the major search engines, in a way that they’d both agree are fair. Nor is our column earlier this week — Google vs. Bing: The Fallacy Of The Superior Search Engine from Conrad Saam — the type of testing I’m talking about.

That column looked at only 20 different queries and found Bing slightly ahead. Pick a different 20, and Google might have “won.” I think you need to run many more queries than that.

In addition, do you discount if either Google or Bing changes the results based on your search history or your location? That might not make sense, if such systems are inherently designed to provide better results for individuals.

Do you discount “smart” answers or vertical search results like news headlines that may appear, focusing only on the “natural” results, those classic “10 blue links?” That’s not how a typical searcher interacts with results.

No One Really Knows

Relevancy testing was complicated enough 10 years ago when results were simpler. Today, figuring out a system is even more complex. Our column wasn’t trying to designate a “winner” in the search sweepstakes but rather point out the fallacy of anyone declaring that one search engine is “superior” to another. The honest answer is that we really don’t know. We don’t have the metrics to assess that.

I’ll be revisiting the relevancy metrics challenge in the future, plus talking with the major search engines about whether there’s a way to go forward with the idea, so that we’re not relying on someone’s ego search or anecdotal accounts to decide which search engine has the best results. There’s got to be a better way than that.

For more on this subject, see some of these past stories from us:

Related Topics: Channel: Strategy | Features: Analysis | Stats: Relevancy | Top News

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • djrobsmith

    I guess one of the problems is that Seo techniques have got so precise that unuseful ‘noise’ is going to show up more. Maybe a more community effort like http://vetosearch.com is a way round it allowing feedback so the search engine can do a better job

  • http://www.blindfiveyearold.com ajkohn

    I tend to agree that overall search quality has gone down. There are a number of theories as to why and how (I’ve got a few myself) but even the idea of relevancy is subjective.

    I may not get much from a Mahalo or Squidoo page, but a lot of people actually do.

    I think the majority of exact match keyword domains are no where near as good as the alternatives. But as you point out – Good Remains Good Enough.

    I’m prone to analogies and the one that continues to crop up in relation to search is food. I see what I think of as more ‘fast food’ coming up higher in SERPs. I, personally, avoid fast food, but I’m NOT the norm. McDonald’s makes a killing, even if there is a ‘better’ alternative down the road.

    So at times I think perhaps we’re seeing a change from the gourmet (those pesky San Francisco food snobs!) to the supermarket. From Whole Foods to Safeway. Of course, I could be completely wrong too.

    As to the apparently conflicting poll data, I think it could be related how the questions are posed. http://www.youtube.com/watch?v=3gMcZic1d4U

  • http://about.me/AndrewMcGarry Andrew McGarry

    I don’t usually comment on articles because I’m not a fan of SEOs clogging up the net with inane opinions just to boost their own site or blog.

    This article doesn’t go far enough to highlight how Google is deliberately trying to increase profits by introducing changes which do not benefit either the user or the SMB advertiser.

    A Google UK search for ‘adobe photoshop’ provides a first six SERPs of Adobe.com – if that isn’t forcing companies to pay more for PPC by squeezing organic, then I don’t know what is.

    Having made it even harder for companies without big SEO budgets to achieve page 1 SERPs for popular terms, they’re now switching their attention to see how they can pull the same trick with PPC.

    Absolute power corrupts absolutely.

    If I was in charge of Bing my message would be simple – give users exactly what they want and don’t make the mistakes Google is making right now.

    Slowly but surely, people will look elsewhere for search results that are not filled with noise from Google’s own services or feature the first 6 SERPs all from the same domain.

    Even if you’re not a fan of Microsoft, Bing is the best alternative right now and the only chance we have of any sort of competition that might just force Google to wise up.

    Microsoft FTW… who would have thunk it?

  • http://rosmarin-search-marketing.com Myron Rosmarin

    From the perspective of a search product manager, the result that only 13% reported not being able to find what they were looking for as the “bigger problem” is like “hitting it out of the park.” To the extent that a user can perform a search and quickly find what they’re looking for is so important that everything else (figuratively and literally) *is* noise. This is why search engines give priority to the problem of getting the first few results as relevant as possible. They’re never going to keep poor quality results out of the SERPs (try as hard as they might, it’s a much harder problem to keep all the poor results out of the search result set than it is to identify the best results and get them to the top). But they don’t have to spend as much time on the problem of filtering out poor results since users don’t care about what’s irrelevant as much as they care about getting to where they want to go.

  • Jose Zurstrassen

    Noise is what killed Altavista back then. Like Google today, they encouraged it at first, but it became unmanageable. http://goo.gl/eNTr8

  • Danny Sullivan

    Andrew, Search Engine Land is a well-respected news site that covers the search and search marketing industries. It’s not a blog run by “a bunch of SEOs clogging up the net with inane opinions.” If that were the case, we wouldn’t be carried by places like Google News, cited by Google or Bing, much less major mainstream news sites from across the web, such as the New York Times or the Wall Street Journal.

    In terms of how Google has changed to display more sites from one source, we covered that ages ago – as well as some pieces against it. But those were just our inane opinions, of course:

    http://searchengineland.com/google-search-results-dominated-by-one-domain-49025

    http://searchengineland.com/google-showing-more-results-per-domain-for-more-queries-56380

  • http://about.me/AndrewMcGarry Andrew McGarry

    Danny,

    >Search Engine Land is a well-respected news site
    I know, that’s why I read it every day.

    >It’s not a blog
    It’s fairly obvious that my comment is about SEOs endlessly plugging *their* own site or blog. Don’t take it personally.

    > we wouldn’t be carried by places like Google News
    Again, no need to defend yourself. However, Google News carries a lot of garbage. I wouldn’t use it as a quality yardstick.

    >how Google has changed to display more sites from one source, we covered that ages ago.
    Yes, and my comment was putting forth the opinion that you could be a bit more critical of Google’s mafia shakedown tactics.

    Keep up the good work.

  • http://www.adrac.co.uk Adrac Ltd

    I believe that Google has been the traget of spammers as opposed to other search engines duw to the amount of traffic it generates. I do believe that recently the Google spam team have been very quite and Google has mainly been concentrating on other search products like Google Places etc..

    I believe that Matt Cutts team needs to pull their socks and improve the quality of results as it’s getting to be annoying.

  • http://managinggreatness.com Gil Reich

    As AJ points out above, I think the way the question was posed affected the lifehacker survey. The question could have been “Do you think Google results have been getting better or worse.” the middle choice could have been “about the same” and the article could have neutral. Instead the article led with how influential bloggers were disappointed that search results had been taken over by “search-optimized sites stuffed with pseudo-content, affiliate links, and overwrought ads” and the question and choices established a negative framework.
    I also suspect more people answer yes than no to “Is X getting worse” for almost any X. I know that music, politicians, and TV shows kept getting better until I was a teenager and then got ruined by big money.
    I have no idea whether or not Google is getting better or worse. The lifehacker survey provides no new information, IMO.

  • http://blog.nextblitz.com galeal

    I would love to see a histogram and analysis on a use case by use case basis. Any suggestions?

    I did a crude analysis last week that showed Google search is not “broken” but would like to work with more granular use cases and quantitative data:

    http://nextblitz.com/blog/google-search-is-not-broken/

    Thanks.

  • http://www.bestshoppingsiteonline.com dogma72

    I dont even use Google anymore. I do alot of research for my work and that takes alot fo searching. Turns out if you use Google too much they will go slower and sometimes say your IP is making too many searches. I understand their need to watch for spammers but its a bit ridiculous.

    So if you have to do research and its alot of research dont use google.

  • Shreshth TanwanI

    The stats are pretty correct, I have tried googling lot of terms and get the result which is not the most accurate.
    Due to this very reason, came up with a social search engine called Bingagain.com
    You can check it at http://www.bingagain.com
    BingAgain is Bing + Facebook combined
    It gives you those results which have been liked by the users at facebook

  • http://sites.google.com/site/deidredrewes Deidre Doom

    I love Google as much as the next person…but have you ever tried finding what you need in blogs? I’m sorry, Matt Cutts, but the spam control on your blog search could really use some work.

    Example: when I search for “fashion shopping” in blogs, expecting fashion bloggers to pop up left and right, I get results like this: http://www.artrocksnewyork.com/
    Blogs that use popular keywords to mask their irrelevant spam (Free online poker, anyone?)

    Maybe it’s just me, but I feel that search is getting too big… as in, too much content to weed through.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide