• Gary Bisha

    They could instead use min freelance and ask the workers to rate a search page result for satisfaction level.

  • http://www.brickmarketing.com/ Nick Stamoulis

    ” A Perfect page should address the intent of at least 50 percent of searchers.”

    I find it interesting that a “perfect” page could still be the wrong information for 1/2 of the searchers. I would have thought that percentage would have been higher but at least Bing recognizes that it’s hard to nail user intent perfectly 100% of the time.

  • ChristianKunz

    I really would like to know the percentage of websites that are rated this way. If the approach is to check a representative share then good look bing!

  • http://www.facebook.com/john.beagle John Beagle

    And how do you go about looking at a billion search result pages and rating them individually for each set of keywords? 

    One thing that might help is if you had a reporting tool for bad serps. 

  • Touseef Hussain

    I think there should be no page 2, page 3, feature.. Bing Should be different. Instead of browsing through pages, there should be some jquery kind of thing which slides the page, it would be a  better experience for the user…We are still using the same method what we were using in 1999

  • totnuckers

    So basically they are just following Google’s lead

  • http://twitter.com/fiend4house internet boss

    They need to focus more on strategic partnerships and effective marketing before this. Their search share continues to tumble against Google. What is a good set of search results worth if nobody ever sees them?

  • http://twitter.com/WesleyLeFebvre Wesley LeFebvre

    Some great insight into how Bing’s search team thinks. Thanks for sharing it with us, Matt!

  • http://twitter.com/goodwinfamily goodwinfamily
  • Iblis Bane

    A 25% chance of the page matching my intent doesn’t sound very excellent to me… (Ok, 25-50%, but at best, that’s still a failure rate of 1 in 2…)

    Are people such sloppy searchers?

  • http://www.facebook.com/the.nathaniel.bailey Nathaniel Bailey

    I get what your saying, but I would have thought there are some good
    reasons behind that “at least” 50% being lower then you might think at
    first.

    Maybe it says “at least 50 percent” for a reason, such as searches that may have two or more meanings, meaning that not all people would find the result helpful or what they are searching for.

  • http://twitter.com/nondisclosure1 non disclosure

    I worked on both Google’s and Bing’s search quality ratings teams. All the stuff you get from these self-proclaimed professional raters you interviewed is beyond basic.

    The stuff you described above is the most basic core principles shared by both teams, and I dare say all similar systems. You don’t even need to be officially hired to get to this information and the chart you showed (copyright?). At both Bing and Google, once you are in the interview process, you’d be sent a handbook that includes all this, but before that, you have to sign a non-disclosure.

    The Google rater you interviewed earlier sounds so incompetent and misunderstood/ failed to explain correctly things so much that the whole article is just misleading.

    Don’t waste your time (and money) interviewing these people in the future. Anyone good enough to understand a non-disclosure agreement would not be dumb enough to talk to you about this. 

  • http://blog.clayburngriffin.com/ Clayburn Griffin

    Sites that unnecessarily delay content loading on purpose, usually to have totally awesome navigation animations, should be marked as bad.

  • DanHigson

    Great post! It amazes me how many people are using Bing. At least it’s not Aol!

  • seoword

    This makes sense. Thank you. 
    My dumb question of the day is what the raters are measuring. Are they rating individual sites or the accuracy of the algorithm for producing quality search results?

  • http://www.netmagellan.com/ Ash Nallawalla

    Here is another way Microsoft evaluates a SERP – Evaluating Search Systems Using Result Page Context – http://www.cs.albany.edu/~ashwin/BaileyIIiX2010.pdf or Learning to Rank for Freshness and Relevance – http://research.microsoft.com/pubs/150747/dai2011.pdf etc Microsoft researchers share their in various academic forums regularly.

  • http://twitter.com/nondisclosure1 non disclosure

    This is what SEL should introduce their readers to, not the “interviews” of self-proclaimed raters, who may not have actually worked for the companies.

    What the raters do is all these theoretical stuff put into practice. There really is no secret.