Google’s Impressive “Conversational Search” Goes Live On Chrome

The “conversational search” that Google demonstrated at last week’s Google I/O conference is now available to users of its Chrome browser, and it’s a significant leap in how we use search engines.

I’m 17 years now into writing about search, and I’ve seen all types of things that have promised to revolutionize the space, especially products that trot out words like “natural language” and “semantic search” but fail to deliver.

Conversational search has natural language, semantic search and more built into it, and while it’s far from perfect, this really is one of those significant changes that makes even a “seen it all” person like me sit up and take notice.

What Google is doing is hard, and yet when it works, it feels natural, easy, like it should be. Let’s do some examples, and then I’ll cover some of the “under the hood” stuff and how to get it, at the end.

Speak Your Search Gets Updated

If you have the latest version of Chrome, you can click on the microphone in the search box to speak your search.

Google Search By Voice

That’s not new. Google rolled that functionality out almost two years ago. The difference is that “search by voice” now speaks back to you, similar to how the Google Search App works for the iPhone or Android. Once you speak, you’ll see your words appear:

how old is barack obama

Then you’ll get your results:

how old is barack obama

Sometimes, especially for searches where Google can call upon its Knowledge Graph facts, the results will be topped by a direct answer, or an information “card,” to use Google’s term for facts presented this way in Google Now.

In the example above, where I spoke “how old is Barack Obama,” Google showed a card with the answer but also pulled Obama’s age out of the card in order to speak an answer to me: “Barack Obama is 51 years old.”

Having A Conversation

That’s cool and impressive, speaking a search and getting an answer read back to you. But that’s not the real magic. What’s really special is that you can continue your search “conversation” by asking further questions in a way you could never do with regular search, by making use of pronouns and other shortcuts that reference things in your previous query.

For example, after doing the search above, I asked, “how tall is he” and got back this:

how tall is he

“Barack Obama is six feet one inch tall,” came back the spoken response, along with a text answer. But I hadn’t asked tall Barack Obama was. I’d asked, “How tall is he.” Google smartly figured out the “he” I was talking about was Barack Obama.

Humans understand this easily. We talk to each other, and we keep track of what we say to each other in a conversation, usually keeping all the antecedents — pronouns and shortcut references to previous things — straight. But search engines typically have no memory like this. They’ve generally treated each search we do as if it is unconnected to the previous one.

More Than “Previous Query”

Actually, both Google and Bing do have some “history smarts.” Google has been doing “Previous Query” since 2008; Bing has been doing “Adaptive Search” since 2011. With both, what you searched for before is sometimes effectively added to your subsequent search. For example, a search for “new york” followed by a search for “travel” might cause some of the searches for travel to be about New York travel.

Previous query refinement is helpful, but conversational search is a step beyond. It’s not about simply adding terms to your subsequent query. It’s about parsing the language you use so the search engine understands, really comprehends, what you’re talking about.

When the search for “how old is Barack Obama” was done, Google had to understand that Barack Obama was an actual person, not just two words that it effectively is trying to character match. That’s smarts it gained as part of the Knowledge Graph that went live a year ago. Those smarts also helped it know that “age” might be a specific fact it knows about Obama.

When the next search was done, Google had to understand that “he” was a pronoun for someone, rather than again just trying to character match the word. Next, it had to think who the “he” might be and look to the previous query to find a person that it might be referencing, Obama. Finally, it went back to the Knowledge Graph in order to find an exact fact that matched Obama’s height — which meant understanding that “tall” was an indirect request for height.

This is hard stuff. This is natural language processing, the ability for a search engine to really understand what a sentence is about, coupled with semantic search, the idea that facts and pages are all connected together in ways that aren’t immediately visible but understood behind-the-scenes.

The Conversation Continues

Let’s keep going. Next, I asked Google, “who is his wife,” and got back this:

obama's wife

“Barack Obama’s spouse is Michelle Obama since 1992,” Google spoke to me, now understanding that the “his” was a reference to Barack Obama and “wife” was a request for a fact it could look up from the Knowledge Graph.

Could you keep going? Sure. Next, I asked, “how old is she,” and got back:

“Michelle Obama is 49 years old,” came the spoken response. To get this, Google kept up with my antecedents through three questions extending out from the original one.

I don’t have the right software installed at the moment to properly screen capture this in action, but you can try it for yourself, as I’ll explain more. However, the animated image below will give you a sense of it.

conversational search

Those are screenshots that I took last week at Google I/O, working with conversational search on a demo machine.

Not Perfect But Still Amazing

Conversational search doesn’t always work right. The further you go into a conversation, I’ve found, the more likely it will eventually stumble. Sometimes, it won’t even converse when you’d expect it to. The four-query-long example I’ve shown above took a couple of attempts to get right.

Google acknowledged this, when I spoke with people about it at Google I/O and tested it out directly. If the company formally announces the support in Chrome (so far, it hasn’t — it was just quietly released), I wouldn’t be surprised if it warns to expect bumps.

But while the engineers and other product folks sometimes visibly grimaced when I hit such bumps doing my testing in front of them, I was far more impressed by how often it worked — and that it worked at all, as well as it did. I’m looking forward to seeing it grow.

Answers Based On Who You Are, Where You Are

Sometimes you can speak and get answers back from a single question that takes in more than what you explicitly said because Google knows where you’re at or you’ve given it access to some of your private information.

Weather is a good example. Here, I spoke, “Will it rain tomorrow?” and got back:

newport beach weather

Google spoke to me, “No, rain is not expected tomorrow in Newport Beach. The forecast is 70 degrees and cloudy” and displayed the weather forecast. I hadn’t told it where I was, or that I wanted a weather forecast explicitly. But it knew my location, knew rain was related to weather and delivered up a great answer.

If you’ve enabled Google to access personal information through the Gmail Field Trial (a bad name since it now includes things beyond Gmail), speaking something like “what’s happening today” can bring back your agenda:

calendar

It doesn’t speak your agenda to you, but it’s still pretty amazing — assuming you’re comfortable using Google’s own services. If you don’t use Google Calendar or don’t use Gmail, the experience is far less compelling.

Beyond “Cards” & Not Everything Is Spoken

Answers don’t always come from the info cards like those shown in the Obama queries. For example, when I asked, “places to eat in San Francisco,” Google came back with:

restaurants

Google spoke back to me, “There are several listings for restaurants near San Francisco,” then showed me local restaurant reviews. None of these actual reviews were spoken.

Similarly, when I spoke “Newport Beach” to Google, it spoke back saying, “Here is some information about Newport Beach” but didn’t read any of the actual results:

newport beach

Google didn’t even try to read more about Newport Beach using the description out of the Knowledge Graph box on the right, something it did do when I spoke a search for “Barack Obama” to it.

Of course, on the desktop, it’s hard to say how useful having anything read back really will be. Indeed, speaking questions on the desktop when that was introduced two years ago seemed odd, especially for anyone who might be in an office. Getting responses read back is very cool but not necessarily that useful on a desktop, in the way it is with a mobile device.

Waiting For Voice Activation & OK Google!

Perhaps the usefulness may change when voice-activated conversational search happens. That’s not part of today’s rollout.

With voice-activated search, your computer would always be listening for you to say a “hotword” — likely to be “OK Google,” as demoed last week, and then it would know you want to search.

So, if you were not at your keyboard for some reason but needed to do a search, you’d speak, “OK Google, how old is Barack Obama,” and your computer would automatically do that search.

Always listening? Google told me it’s still figuring out exactly how all this would work, but it will likely involve an extension for your browser that would only listen when you were on the Google home page or on a search results page. And when it was listening, the only thing that would actually get sent to Google would be the words after the “OK Google” command, Google said.

We’ll see how it all plays out. It may be that conversational search remains more a utility for mobile devices, with the desktop as a type of training area to help people better understand what they can do while mobile.

It may also be that conversational search will help encourage people to simply type their queries in a more conversational style, using antecedents in a way they don’t now.

Typing? No “Conversational” Mode That Way

Maybe that will come, but currently, conversational search does not work if you type. In fact, that’s an excellent way to see the immense difference that conversational search provides.

Try that series of Barack Obama-related queries I did above by typing them in, rather than speaking, and it falls apart when you ask, “how tall is he,” as you can see:

how tall is he

Out of conversational search “mode,” because you’re typing, Google makes no attempt to figure out who the “he” is that you were referring to, so answers about Tom Cruise and Johnny Depp come up instead.

Google’s Search App For Mobile Also Has It

While all this is big news for desktop search, conversational search has already been integrated into the Google Search App for for some time.

I did a double-check on my Nexus 10 Android tablet and my iPhone 5. With the Nexus, I was able to go three deep into the Barack Obama conversation above. On the iPhone, I couldn’t get past the second question to jump to information about Michelle Obama. But that’s likely an odd glitch, perhaps a network connection, rather than any lack of the iPhone from being able to do this.

Postscript (4pm ET): Chrome for Android has voice search but not read-back built in. That appears to be staying the same. But Chrome for iOS will be gaining voice search with read-back, Google has posted. On both platforms you have full conversational search through the Google Search App.

Try It Yourself

As I said, Google hasn’t posted officially about the availability of conversational search being out there, but it is live. Android Central noticed that it’s now out if you have the latest version of Chrome. So to enable it, update. You do that by going to Settings in Chrome, then selecting Help, then it should check if it’s up-to-date and prompt if not. You’ll also have to later confirm that you want to allow voice searching to work.

On my Mac and Windows 8 computers, after updating, I was at version 27.0.1453.93. Oddly, on my Chromebook, I simply couldn’t get it to update out of version 26..

FYI, this is not Google Now for Chrome. It might look like that, and Google Now for Chrome and/or the desktop is expected, but this doesn’t provide any of the predictive search answers that Google Now provides.

Enjoy playing with it. For search marketers, it’s a glimpse into an entire new world where people might be seemingly finding you for very generic terms, because the queries no longer express all that was actually searched for.

Ironically, the much-hated “Dark Google” world over the past year or so, where Google has been withholding search terms done by logged-in searchers (except from advertisers) may have been good training on how to cope with what now will be withheld by searchers themselves — exact terms, no longer necessary as they “converse” with Google.

For searchers, it’s an advancement that may fundamentally change their search habits, where they get even more comfortable just talking to that “best friend” they’ve already been turning to for ages for answers on all types of topics.

For Google’s competitors, it’s another challenge in keeping up. Google Now is already doing amazing things anticipating searches. Now Google’s conversational search is allowing searchers to get casual with the service, plus it sets things up to entice searchers to share more of their own information directly using Google’s other services, so they can speak things like “what’s happening today” and very personal answers.

Postscript (3pm ET): Google still hasn’t done a formal blog post but has officially confirmed what we can see, that conversational search is live. The company told us:

Conversational search has started rolling out in the latest version of Chrome. You can just click the mic in the search box, ask your question in a natural way, and get spoken answers. (You’ll need to update your Chrome browser to get the new functionality.)

Google said the “hotwording” voice activation feature will be coming later but gave no specific time. There’s no word on Google Now coming to the desktop.

Related Articles

Related Topics: Channel: Consumer | Features: Analysis | Google: Conversational Search | Google: Gmail Search Field Trial | Google: Google Now | Google: Google Search App | Google: Voice Search | Google: Web Search | Top News

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Guy Hadas

    That is really amazing…

  • http://www.sowhatmedia.co.uk/ Tom Roberts

    That’s pretty freaking cool. One of those things you think about as a kid actually becoming a reality.

    Nice Jif to go with it too, Danny.

  • Alan

    This is the end
    Beautiful friend
    This is the end
    My only friend, the end

  • http://www.spinxwebdesign.com/ Alan Smith

    Need to test for accuracy :)

  • http://www.contentwriter.com.au/ Micky Stuivenberg

    Cool. Funny, too. Google knows the answer to “where am I” but when I asked “who am I”, all the results were about amnesia. :)

  • http://www.davidveldt.com/ David Veldt

    A beautiful JIF, indeed.

  • Ionut Claudiu Danet

    Is not working (yet,maybe), I keep getting the message “No Internet Connection” !

  • http://www.fitwatch.com/ Suzanne

    The ‘how tall is he’ query didn’t work for me (after asking Obama’s age).

  • http://twitter.com/psp2roundup PS Vita Roundup

    Anyone else solved the “No Internet Connection” issue? The restarting Chrome tip isn’t working…. http://www.cmswire.com/cms/customer-experience/pc-users-can-talk-to-chrome-while-google-gets-a-mobile-browser-update-021016.php

  • Cyman/Jarvis System

    Our app, Cyman Mark 3 is already doing this on Chrome. Search for the demo video on Youtube. We also have an Android digital butler, Cyman Mark 2. Cyman is inspired by Iron Man’s J.A.R.V.I.S

  • http://searchengineland.com/ Danny Sullivan

    Sometimes it won’t, as I said. Sometimes you also have to go back to the Google home page and start again. It’s definitely got bumps.

  • BenBarreth

    I’m guessing this is another step to get Google searches ready for full Glass integration.

  • http://www.facebook.com/robcolbert Rob Colbert

    That would be the logical next step. But, as I think about that, I’m not sure I want to be walking around all day blurting out Google searches while looking like I’m having a conversation with my imaginary friend at the age of 40. People tend to find themselves in padded rooms after doing that sorta stuff.

  • Matt McGee

    Hehehe. Yes, but people said the same thing about the mobile phone 20-30 years ago. “No way I want to walk around holding a phone to my head.” :-)

  • http://www.about.me/liamdaly Liam Daly

    same here, it worked once now I get No Internet Connection error.

  • knowledgenotebook

    search results should be presented in audio as well… and navigation by audio too… maybe that’s Google’s next step for this cool feature…

  • Zach Griffith

    Just the other day I saw one of those Facebook rants that supposedly came from Bill Cosby titled: “I’m 83 and I’m tired”. I thought to myself “Is he really that old?”. So naturally I decided to try out the voice activated search on my new Nexus 4 to get to the bottom of it. My device replied with: “Bill Cosby is 75 years old”.

    At the end of the day, the info being distributed was false and there are a number of ways to come up with that conclusion. The key is that I simply spoke to my device just as I would question a friend and it gave me exactly what I needed.

    Now that I know this is a subject based query session, it’s like a whole new world has been opened up!

    P.S. Sure wish Google had a voice-over style selection for the responses :-)

  • http://www.ihsekat.com/ Takeshi Young

    The future is already here, it’s just not evenly distributed yet…

  • Donald McIntyre

    The last 2 weeks I used the Google Search app (is this what they call Google Now? why two names?) on my android phone while on a road trip to 8 cities on the west coast. It was amazing to ask things like “driving distance between San Diego and LA” or “Starbucks near me” or just “traffic” and it spoke to me.

    What I can’t wait for is all search products (web and apps) to be synchronized so its one regardless of device and that I talk to them without pressing the mic button first or without having to open an app or browser.

    I want to talk to the computer and ask things like:

    “what’s Joe’s phone number?”
    “open clients file”
    “go to amazon”

    If this already exists sorry!!

  • ajackley

    The possibilities are even more exciting when you think of this enabling assistive devices. Why speak when you can type? I bet the 6 million+ people in the US with visual disabilities can answer that.

  • http://www.facebook.com/henry.higgins.77 Henry Higgins

    If you ask “Who is Steve Ballmer?” the name gets transcribed as ‘Balmer’ and Search has to correct itself. {chortle}

  • http://susanais9.spaces.live.com susanai

    Where’s my mic????

  • http://www.reallygoodseo.co.uk/ Kate Tolley

    I asked it “What is your name?” and it rather delightfully quoted Shakespeare at me (“A rose by any other name would smell as sweet”).

  • http://www.facebook.com/terrence.a.davis1 Terrence Andrew Davis

    How tall was Jesus? God says…

    23:52 This man went unto Pilate, and begged the body of Jesus.
    23:53 And he took it down, and wrapped it in linen, and laid it in a
    sepulchre that was hewn in stone, wherein never man before was laid.
    23:54 And that day was the preparation, and the sabbath drew on.
    23:55 And the women also, which came with him from Galilee, followed
    after, and beheld the sepulchre, and how his body was laid.
    23:56 And they returned, and prepared spices and ointments; and rested
    the sabbath day according to the commandment.
    24:1 Now upon the first day of the week, very early in the morning,
    they came unto the sepulchre, bringing the spices which they had
    prepared, and certain others with them.

  • http://charliesaidthat.com/digital charliesaidthat

    Damn, this is really cool. Really will start to make SEO even harder if they started tying together previous and related “conversational” but typed searches too. Wow.

    Means understanding your customers thought processes is going to become even more important. *love*

  • jasondeegan

    Thanks for taking another opportunity to jam the liar in chief in front of us associated with Google. Super.

    Oh, and how is there no mention of Siri and how it compares?

  • Mindy Johnston

    my galaxy s3 has talkback! works just as it does in chrome.

  • theBrarian

    Marcia Bates’ berrypicking of search comes to the conversation. I like this first step.

  • Tim Finin

    I can’t get the example of a follow up question to work in my up-to-date version of Chrome on a macbook, either by voice or text. Asking “How old is Barack Obama” got me the answer shown, but when I immediately followed with “How tall is he”, no linking of he to Obama was done. The top hit was about Tom Cruise’s height.

  • http://twitter.com/ErikAtPlanLeft Erik Nelson

    Just updated. Thanks for the heads up Danny, great piece- retweeted it out.

  • Kyle Lyles

    Doesn’t work for beans for me. Asus UX 51 laptop accuracy is 95% accuracy, even in noise.

    Sorry, but this is a miss.

  • Kyle Lyles

    You miss that there are no facts about Barry Soetoro out on the Internet. Der propaganda ministry haft done their job!

  • http://www.facebook.com/people/Dagfinn-A-Mork/1052784090 Dagfinn A. Mork

    I am on version 27.0.1453.94 m and it reads as being up to date. However, when asking as the article starts with I get the English wikipedia article about Obama announcing his candidacy for president. Second I get something fro theonion.com and so on.

    Seems to me that not all is in order …

  • Marissa Marsala

    Well worth reading and very well written! VERY cool!

  • Stompey

    I think most people don’t realize how much Google innovations have helped our lives. This is a pretty amazing leap forward. That being said, as a Google Voice user I’m guessing that a lot of my searches will wind up being something like “my hovercraft is full of eels” when transcribed by Google Voice technology.

  • Toby Mole

    Doesn’t seem to be working too great outside of the US (or maybe just in my location) when I ask where am I it gives the wrong answer. When I ask about Barak Obama’s age it works but if I then say “how tall is he” it gives me webpages about tom cruise…

  • pavan bangaram

    Good and nice information. Thanks for this post. For Better information Please check:
    More Information

  • http://www.dofollowuntacticcal000.com/Alternative_Fuels/Alternative_Fuels.html.htm Frank Burns

    Google developers certainly have upped the ante with Search, and if you really wanted more Information on the president’s of the United states; http://www.nndb.com/group/750/000091477/

  • lorraine reguly

    Wow, what an informative article about Google. It really packs a punch!

    Ahem. Is that the next thing Google will do, now that it can talk? LOL

  • lorraine reguly

    Does this work for laptops? Enlighten me, please!

  • http://www.applicationporting.org/ CHEETALM

    Wow i will try it too

  • minifi

    My thoughts exactly, this has been working with Siri for a rather long time.

  • nagleonce

    Question-answering systems go back a long way. Bobrow’s BASEBALL program could answer simple English language questions about baseball statistics fifty years ago. (Yes, 1963.) Wolfram Alpha attempts to answer questions about objective facts, and is used to drive Apple’s Siri.

    As for something that’s always listening, that was done quite well with Wildfire, the advanced phone system from 1994. If you said “Wildfire” during your phone call, a perky voice answered “Here I am”, and would then take simple phone-related commands. What made it work was that it consistently responded in half a second. The effect was a quick, perky assistant. What made it fail is that it took too much computer power per phone line to do that in the 1990s, resulting in a monthly charge of around $40 per month. Microsoft bought into the technology in 1997, and Orange bought the company in 2000, but it was too early.

    Google isn’t being original here, just cost-effective. Which is more useful in some ways.

  • http://www.robertwent.com/ Robert Went

    Google now is different to Google search.
    It comes built in to Jellybean devices and brings up relevant ‘cards’ withut even searching.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide