• Search Engine Land
  • Sections
    • SEO
    • SEM
    • Local
    • Retail
    • Google
    • Bing
    • Social
    • Resources
    • More
    • Home
  • Follow Us
    • Follow
  • Search Engine Land
  • SEO
  • SEM
  • Local
  • Retail
  • Google
  • Bing
  • Social
  • Resources
  • Live
  • More
  • Events
    • Follow
  • SUBSCRIBE

Search Engine Land

Search Engine Land
  • SEO
  • SEM
  • Local
  • Retail
  • Google
  • Bing
  • Social
  • Resources
  • More
  • Newsletters
  • Home
Mobile

Google (Now) Beats Siri And Cortana For Direct Answers — Study

Stone Temple Consulting (STC) published a study in which it compared mobile answers to “3086 different queries” on Siri, Google and Cortana to determine how each performed relative to the others. Google was the winner with more direct answers and more complete information. Eric Enge, who wrote up the results, says this was not a “personal assistant […]

Greg Sterling on October 8, 2014 at 11:58 am
  • More
google-now-fade-1920

Stone Temple Consulting (STC) published a study in which it compared mobile answers to “3086 different queries” on Siri, Google and Cortana to determine how each performed relative to the others. Google was the winner with more direct answers and more complete information.

Eric Enge, who wrote up the results, says this was not a “personal assistant comparison.” Rather, it was a “knowledge box” or “knowledge panel” comparison. We’ll instead refer to these as “answers,” which would be the layperson’s view of the information.

The queries tested were mostly factual, such as “What’s the tallest mountain in the world?,” “How long is the Gettysburg Address?” and “When is the next Stanley Cup?” Although there were occasional opinion questions such as “What is love?”

I’m not going to discuss the study methodology in depth. One might critique it in various ways, but I accept the overall findings as valid. Below is a summary of the major findings.

STC noted that Google returns more enhanced results vs. Cortana and Siri when asked questions expected to produce such a result (“knowledge box”).

STC knowledge box showdown

Google also bested the other two in terms of providing most direct and complete answers to fact-based queries. In nearly 90 percent of instances, Google did a better job of clearly and directly answering the query posed. STC said that it was very strict in grading this portion of the test and didn’t accept as correct answers that partially or indirectly responded to the question.

STC completeness Google Now vs. Siri, Cortana

One of the more interesting findings reports that in cases where Google is presenting an “answer,” STC says that it only gives third-party attribution 25 percent of the time. The other 75 percent of answers had no source.

In summary, STC concludes that Google “has a clear lead in terms of the sheer volume of queries addressed, and more complete accuracy with its queries than either Siri or Cortana … [and that] Google is progressing the fastest on all fronts.”

Siri is not a search engine but a combination of natural language understanding (NLU) technology and “artificial intelligence” to help deliver third-party information. There is no Apple index to draw from; Siri “routes” queries to information or answers — including web search results from Bing. Indeed, it’s somewhat ironic that Microsoft is aggressively marketing Cortana (vs. Siri) to promote Windows Phones because Bing is the source of Siri’s web search results.

The original vision for Siri was a smart voice front end tied to multiple structured data sources on the back end to completely bypass the SERP in mobile and yield “actionable” information. Siri’s arrival a few years ago was one reason Google has pushed deeper into structured information — in order to provide more “answers” in mobile. That and the general movement away from “10 blue links” also motivated Google to put more answer boxes and knowledge graph results on PC SERPs.

Siri has not lived up to its promise, and the question for Apple is how much of a search arms race it wants to get into. Regardless, it needs to “fix” Siri. For its part, Microsoft intends to make Cortana an interface across platforms and devices, including Xbox. The personal assistant metaphor and the provision of “answers” are slowly taking over from basic search and conventional results.

The movement toward answers is also a major source of publisher frustration and the sense of “betrayal” at the heart of much of the recent anti-competitive criticism of Google.


Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.



About The Author

Greg Sterling
Greg Sterling is a Contributing Editor to Search Engine Land, a member of the programming team for SMX events and the VP, Market Insights at Uberall.

Related Topics

Apple: SiriBing MobileBing Voice SearchChannel: MobileGoogle: Featured Snippets & Direct AnswersGoogle: Google NowGoogle: MobileGoogle: User InterfaceGoogle: Voice SearchMicrosoft: CortanaSearch Features: Direct Answers

We're listening.

Have something to say about this article? Share it with us on Facebook, Twitter or our LinkedIn Group.

Get the daily newsletter search marketers rely on.
See terms.

ATTEND OUR EVENTS

Lorem ipsum doler this is promo text about SMX events.

February 23, 2021: SMX Report

April 13, 2021: SMX Create

May 18-19, 2021: SMX London

June 8-9, 2021: SMX Paris

June 15-16, 2021: SMX Advanced

August 17, 2021: SMX Convert

November 9-10, 2021: SMX Next

October 2021: SMX Advanced Europe

December 14, 2021: SMX Code

Available On-Demand: SMX

×


Learn More About Our SMX Events

Discover actionable tactics that can help you overcome crucial marketing challenges. Our next conference will be held:

MarTech 2021: March 16-17

MarTech 2021: Sept. 14-15

MarTech 2020: Watch On-Demand

×

Attend MarTech - Click Here


Learn More About Our MarTech Events

White Papers

  • Become an Experience-Led Business with an Agile CX Tech Stack
  • Transformational CX: Video-Powered Experiences Meet Consumers’ Demands
  • What Customer Experience Means in 2021
  • 4 Ways to Commodity-Proof Your Company
  • Threat report: How are different industries battling cybercrime?
See More Whitepapers

Webinars

  • The AI Revolution Is Coming to Every Stage of Your Buyer’s Journey
  • The Fundamentals of Link Building for E-Commerce & Affiliate Sites in 2021
  • Your Customer is Calling: Make the Most of Your Marketing Spend with Call Tracking
See More Webinars

Research Reports

  • Local Marketing Solutions for Multi-Location Businesses
  • Enterprise Digital Asset Management Platforms
  • Identity Resolution Platforms
  • Customer Data Platforms
  • B2B Marketing Automation Platforms
  • Call Analytics Platforms
See More Research

Attend SMX For Only $99

h
Receive daily search news and analysis.

Channels

  • SEO
  • SEM
  • Local
  • Retail
  • Google
  • Bing
  • Social

Our Events

  • SMX
  • MarTech

Resources

  • White Papers
  • Research
  • Webinars

About

  • About Us
  • Contact
  • Privacy
  • Marketing Opportunities
  • Staff

Follow Us

  • Facebook
  • Twitter
  • LinkedIn
  • Newsletters
  • RSS
  • Youtube

© 2021 Third Door Media, Inc. All rights reserved.

Your privacy means the world to us. We share your personal information only when you give us explicit permission to do so, and confirm we have your permission each time. Learn more by viewing our privacy policy.Ok