• Search Engine Land
  • Sections
    • SEO
    • SEM
    • Local
    • Retail
    • Google
    • Bing
    • Social
    • Resources
    • More
    • Home
  • Follow Us
    • Follow
  • Search Engine Land
  • SEO
  • SEM
  • Local
  • Retail
  • Google
  • Bing
  • Social
  • Resources
  • Live
  • More
  • Events
    • Follow
  • SUBSCRIBE

Search Engine Land

Search Engine Land
  • SEO
  • SEM
  • Local
  • Retail
  • Google
  • Bing
  • Social
  • Resources
  • More
  • Newsletters
  • Home
SEO

The Google Quality Raters Handbook

Brian Ussery has discovered a revised copy of the Google Quality Raters Guidelines, which he archived on his own site. The documents are used by Google Quality Raters to aid them in classifying queries, measuring relevancy, and rating the search results. To do so, the Quality Rater must understand how Google works and this document […]

Barry Schwartz on March 14, 2008 at 9:30 am
  • More

Brian Ussery has discovered a revised copy of the Google Quality Raters Guidelines, which he archived on his own site.

The documents are used by Google Quality Raters to aid them in classifying queries, measuring relevancy, and rating the search results. To do so, the Quality Rater must understand how Google works and this document has a bunch of that. Let me pull out some of those details in easy to read bullet points.


Three Query Types:

  • Navigational: someone searching for a site, such as a search for IBM.
  • Informational: someone searching for information on a topic of interest, such as finding out more information on Danny Sullivan.
  • Transactional: someone searching when seeking to purchase something either online or offline, such as searching for ‘buy ipod touch.’

Quality Rating Scales:

  • Vital: This is the highest score a web page can receive for a query. A vital result comes from a query that is most likely navigational and the resulting page is the official web page of the query. When searching for ‘ibm’, the vital result would be www.ibm.com.
  • Useful: This is the second highest score a web page can receive for a given query. A useful rating should be assigned to results that “answer the query just right; they are neither too broad nor too specific.” One of the examples given for a useful rating would be a search on meningitis symptoms with a resulting web page of http://www.webmd.com/hw/infection/aa34586.asp
  • Relevant: This comes after a useful rating, and is used for results that return less useful results. The guidelines say the result is often “less comprehensive, come from a less authoritative source, or cover only one important aspect of the query.” An example would be a review of laptop computers, but the review only takes five computers and not all computers within its class. Since it is not a fully comprehensive review, it would be rated as relevant and not useful.
  • Not Relevant: This rating is used for pages that are not helpful to the query but are somewhat still connected to the original query. Classifications of a not relevant page would be “outdated, too narrowly regional, too specific, too broad” and so on. One of the examples give is a search for the ‘BBC’ that returns a specific article from BBS; it is too specific and is not relevant to the query at hand.
  • Off-Topic: This is the lowest rating a page can receive for a query. If the returned page is completely not relevant to the query, it would be given a rating of “off topic.” An example given is a query on ‘hot dogs’ that returns a page about doghouses.

Categories For Results That Can’t Be Rated:
Not everything can be rated, and those must be classified somehow. The categories for those types of results include:

  • Didn’t Load: For pages that return a 404 error, page not found, product not found, server time out, 403 forbidden, login required, and so on.
  • Foreign Language: This is given to a page that is in a “foreign language” to the “target language” of the query. English is never a foreign language, no matter what. So, if you search in Chinese for something and a Hebrew page is returned, it is a foreign language, but if an English page is returned, it is not a foreign language. There are exceptions to the rule.
  • Unratable: When the rater cannot rate it for any other reason.

Spam Labels:
Now for the really good stuff, spam labels. This is a new addition to the quality raters guidelines and is fairly small. The labels include:

  • Not Spam: The not spam rating is given to pages that “has not been designed using deceitful web design techniques.”
  • Maybe Spam: This label is given when you feel the page is “spammy,” but you are not 100% convinced of that.
  • Spam: Given to pages you feel are violating Google’s webmaster guidelines.

Flags:
Flags are for pages that require immediate attention, such as:

  • Pornographic content
  • Malicious code on pages

That is a brief overview of some of the many points in the document. For more, see the archived document and for some history, check out Google Blogoscoped. Here is an additional copy of this document at Huomah.com.



About The Author

Barry Schwartz
Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry's personal blog is named Cartoon Barry and he can be followed on Twitter here.

Related Topics

Channel: SEOGoogle: Search Quality RatersGoogle: SEOGoogle: Web SearchSEO: General

We're listening.

Have something to say about this article? Share it with us on Facebook, Twitter or our LinkedIn Group.

Get the daily newsletter search marketers rely on.
See terms.

ATTEND OUR EVENTS

Lorem ipsum doler this is promo text about SMX events.

February 23, 2021: SMX Report

April 13, 2021: SMX Create

May 18-19, 2021: SMX London

June 8-9, 2021: SMX Paris

June 15-16, 2021: SMX Advanced

August 17, 2021: SMX Convert

November 9-10, 2021: SMX Next

October 2021: SMX Advanced Europe

December 17, 2021: SMX Code

Available On-Demand: SMX

×


Learn More About Our SMX Events

Discover actionable tactics that can help you overcome crucial marketing challenges. Our next conference will be held:

MarTech 2021: March 16-17

MarTech 2021: Sept. 14-15

MarTech 2020: Watch On-Demand

×

Attend MarTech - Click Here


Learn More About Our MarTech Events

White Papers

  • Data SEO – The Next Big Adventure
  • Getting Started with Email Marketing Automation
  • The State of Local Marketing Report 2020-2021
  • Quality CRM Data: The Key to Delivering Great Customer Experiences
  • How the Microsoft Search Network Can Maximize Your Search Campaigns
See More Whitepapers

Webinars

  • How to Avoid the Digital Transformation Trap
  • How to Build a Marketing System of Record
  • Meet BIMI: The brand-boosting email security marketers must have for 2021
See More Webinars

Research Reports

  • Local Marketing Solutions for Multi-Location Businesses
  • Enterprise Digital Asset Management Platforms
  • Identity Resolution Platforms
  • Customer Data Platforms
  • B2B Marketing Automation Platforms
  • Call Analytics Platforms
See More Research

h
Receive daily search news and analysis.
Search Engine Land
Download the Search Engine Land App on iTunes Download the Search Engine Land App on Google Play

Channels

  • SEO
  • SEM
  • Local
  • Retail
  • Google
  • Bing
  • Social

Our Events

  • SMX
  • MarTech

Resources

  • White Papers
  • Research
  • Webinars
  • Search Marketing Expo
  • MarTech Conference

About

  • About Us
  • Contact
  • Privacy
  • Marketing Opportunities
  • Staff
  • Connect With Us

Follow Us

  • Facebook
  • Twitter
  • LinkedIn
  • Newsletters
  • Instagram
  • RSS
  • Youtube
  • iOS App
  • Google Play

© 2021 Third Door Media, Inc. All rights reserved.

Your privacy means the world to us. We share your personal information only when you give us explicit permission to do so, and confirm we have your permission each time. Learn more by viewing our privacy policy.Ok