New search quality raters guidelines for Google Assistant and voice search evaluations

Google has published on the Google Research blog the search quality raters guidelines, contractors guidelines to evaluate Google’s search results, specifically for the Google Assistant and voice search results. It is similar to the web search quality guidelines, but it changes in that there is no screen to look at when evaluating such results; instead you are evaluating the voice responses from the Google Assistant.

Google explained, “The Google Assistant needs its own guidelines in place, as many of its interactions utilize what is called ‘eyes-free technology,’ when there is no screen as part of the experience.” Google has designed machine learning and algorithms to try to make the voice responses and “answers grammatical, fluent and concise.” Google said that they ask raters to make sure that answers are satisfactory across several dimensions:

The short, only seven-page, guidelines can be downloaded as a PDF over here.


About The Author

Barry Schwartz
Barry Schwartz is Search Engine Land's News Editor and owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on social media at @rustybrick, +BarrySchwartz and Facebook. For more background information on Barry, see his full bio and disclosures, click over here.