Google’s Matt Cutts On How They Evaluate New Search Algorithms

matt-cutts-algo-change-evalGoogle Head of Search Spam Matt Cutts posted a video today answering how Google goes about evaluating which new search algorithms they use and which they throw away or adapt.

The question was posed by James Foster of Sydney, Australia who asked:

What are some of the metrics that Google uses to evaluate whether one iteration of the ranking algorithm is delivering better quality results to users than another?

Matt Cutts breaks it down to about three steps of the evaluation process:

(1) They test the algorithm offline, benchmarking how the results rank with the new algorithms and if the URLs are higher quality than the previous algorithms in place. The quality is based on how the search quality raters rate the URLs in previous cases. If the URLs were unrated, Google can request these raters to rate the new URLs or compare the old search results to this new test set. Then based on those metrics, Google may decide to move the test to the next phase.

(2) Live tests, where Google will sample a subset of real live searchers and give them the new results with the new set of test algorithms. If Google sees a higher click rate on the new search results, it may imply that the new results are better than the older ones. This is not always the case, specifically with webspam, Cutts said. But in general, the more clicks on a specific search result page, the better quality the results.

(3) Then the Google Search Quality Launch Committee has the ultimate say on if the algorithm goes live to the public or not.

Matt said Google has this down to a “pretty good system” but every now and then they need to refine some of the processes within this workflow.

Here is the video:

Related Topics: Channel: SEO | Google: SEO | Google: Web Search | Top News


About The Author: is Search Engine Land's News Editor and owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry's personal blog is named Cartoon Barry and he can be followed on Twitter here. For more background information on Barry, see his full bio over here.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.

Comments are closed.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide