Google’s Impressive “Conversational Search” Goes Live On Chrome
The “conversational search” that Google demonstrated at last week’s Google I/O conference is now available to users of its Chrome browser, and it’s a significant leap in how we use search engines.
I’m 17 years now into writing about search, and I’ve seen all types of things that have promised to revolutionize the space, especially products that trot out words like “natural language” and “semantic search” but fail to deliver.
Conversational search has natural language, semantic search and more built into it, and while it’s far from perfect, this really is one of those significant changes that makes even a “seen it all” person like me sit up and take notice.
What Google is doing is hard, and yet when it works, it feels natural, easy, like it should be. Let’s do some examples, and then I’ll cover some of the “under the hood” stuff and how to get it, at the end.
Speak Your Search Gets Updated
If you have the latest version of Chrome, you can click on the microphone in the search box to speak your search.
That’s not new. Google rolled that functionality out almost two years ago. The difference is that “search by voice” now speaks back to you, similar to how the Google Search App works for the iPhone or Android. Once you speak, you’ll see your words appear:
Then you’ll get your results:
Sometimes, especially for searches where Google can call upon its Knowledge Graph facts, the results will be topped by a direct answer, or an information “card,” to use Google’s term for facts presented this way in Google Now.
In the example above, where I spoke “how old is Barack Obama,” Google showed a card with the answer but also pulled Obama’s age out of the card in order to speak an answer to me: “Barack Obama is 51 years old.”
Having A Conversation
That’s cool and impressive, speaking a search and getting an answer read back to you. But that’s not the real magic. What’s really special is that you can continue your search “conversation” by asking further questions in a way you could never do with regular search, by making use of pronouns and other shortcuts that reference things in your previous query.
For example, after doing the search above, I asked, “how tall is he” and got back this:
“Barack Obama is six feet one inch tall,” came back the spoken response, along with a text answer. But I hadn’t asked tall Barack Obama was. I’d asked, “How tall is he.” Google smartly figured out the “he” I was talking about was Barack Obama.
Humans understand this easily. We talk to each other, and we keep track of what we say to each other in a conversation, usually keeping all the antecedents — pronouns and shortcut references to previous things — straight. But search engines typically have no memory like this. They’ve generally treated each search we do as if it is unconnected to the previous one.
More Than “Previous Query”
Actually, both Google and Bing do have some “history smarts.” Google has been doing “Previous Query” since 2008; Bing has been doing “Adaptive Search” since 2011. With both, what you searched for before is sometimes effectively added to your subsequent search. For example, a search for “new york” followed by a search for “travel” might cause some of the searches for travel to be about New York travel.
Previous query refinement is helpful, but conversational search is a step beyond. It’s not about simply adding terms to your subsequent query. It’s about parsing the language you use so the search engine understands, really comprehends, what you’re talking about.
When the search for “how old is Barack Obama” was done, Google had to understand that Barack Obama was an actual person, not just two words that it effectively is trying to character match. That’s smarts it gained as part of the Knowledge Graph that went live a year ago. Those smarts also helped it know that “age” might be a specific fact it knows about Obama.
When the next search was done, Google had to understand that “he” was a pronoun for someone, rather than again just trying to character match the word. Next, it had to think who the “he” might be and look to the previous query to find a person that it might be referencing, Obama. Finally, it went back to the Knowledge Graph in order to find an exact fact that matched Obama’s height — which meant understanding that “tall” was an indirect request for height.
This is hard stuff. This is natural language processing, the ability for a search engine to really understand what a sentence is about, coupled with semantic search, the idea that facts and pages are all connected together in ways that aren’t immediately visible but understood behind-the-scenes.
The Conversation Continues
Let’s keep going. Next, I asked Google, “who is his wife,” and got back this:
“Barack Obama’s spouse is Michelle Obama since 1992,” Google spoke to me, now understanding that the “his” was a reference to Barack Obama and “wife” was a request for a fact it could look up from the Knowledge Graph.
Could you keep going? Sure. Next, I asked, “how old is she,” and got back:
“Michelle Obama is 49 years old,” came the spoken response. To get this, Google kept up with my antecedents through three questions extending out from the original one.
I don’t have the right software installed at the moment to properly screen capture this in action, but you can try it for yourself, as I’ll explain more. However, the animated image below will give you a sense of it.
Those are screenshots that I took last week at Google I/O, working with conversational search on a demo machine.
Not Perfect But Still Amazing
Conversational search doesn’t always work right. The further you go into a conversation, I’ve found, the more likely it will eventually stumble. Sometimes, it won’t even converse when you’d expect it to. The four-query-long example I’ve shown above took a couple of attempts to get right.
Google acknowledged this, when I spoke with people about it at Google I/O and tested it out directly. If the company formally announces the support in Chrome (so far, it hasn’t — it was just quietly released), I wouldn’t be surprised if it warns to expect bumps.
But while the engineers and other product folks sometimes visibly grimaced when I hit such bumps doing my testing in front of them, I was far more impressed by how often it worked — and that it worked at all, as well as it did. I’m looking forward to seeing it grow.
Answers Based On Who You Are, Where You Are
Sometimes you can speak and get answers back from a single question that takes in more than what you explicitly said because Google knows where you’re at or you’ve given it access to some of your private information.
Weather is a good example. Here, I spoke, “Will it rain tomorrow?” and got back:
Google spoke to me, “No, rain is not expected tomorrow in Newport Beach. The forecast is 70 degrees and cloudy” and displayed the weather forecast. I hadn’t told it where I was, or that I wanted a weather forecast explicitly. But it knew my location, knew rain was related to weather and delivered up a great answer.
If you’ve enabled Google to access personal information through the Gmail Field Trial (a bad name since it now includes things beyond Gmail), speaking something like “what’s happening today” can bring back your agenda:
It doesn’t speak your agenda to you, but it’s still pretty amazing — assuming you’re comfortable using Google’s own services. If you don’t use Google Calendar or don’t use Gmail, the experience is far less compelling.
Beyond “Cards” & Not Everything Is Spoken
Answers don’t always come from the info cards like those shown in the Obama queries. For example, when I asked, “places to eat in San Francisco,” Google came back with:
Google spoke back to me, “There are several listings for restaurants near San Francisco,” then showed me local restaurant reviews. None of these actual reviews were spoken.
Similarly, when I spoke “Newport Beach” to Google, it spoke back saying, “Here is some information about Newport Beach” but didn’t read any of the actual results:
Google didn’t even try to read more about Newport Beach using the description out of the Knowledge Graph box on the right, something it did do when I spoke a search for “Barack Obama” to it.
Of course, on the desktop, it’s hard to say how useful having anything read back really will be. Indeed, speaking questions on the desktop when that was introduced two years ago seemed odd, especially for anyone who might be in an office. Getting responses read back is very cool but not necessarily that useful on a desktop, in the way it is with a mobile device.
Waiting For Voice Activation & OK Google!
Perhaps the usefulness may change when voice-activated conversational search happens. That’s not part of today’s rollout.
With voice-activated search, your computer would always be listening for you to say a “hotword” — likely to be “OK Google,” as demoed last week, and then it would know you want to search.
So, if you were not at your keyboard for some reason but needed to do a search, you’d speak, “OK Google, how old is Barack Obama,” and your computer would automatically do that search.
Always listening? Google told me it’s still figuring out exactly how all this would work, but it will likely involve an extension for your browser that would only listen when you were on the Google home page or on a search results page. And when it was listening, the only thing that would actually get sent to Google would be the words after the “OK Google” command, Google said.
We’ll see how it all plays out. It may be that conversational search remains more a utility for mobile devices, with the desktop as a type of training area to help people better understand what they can do while mobile.
It may also be that conversational search will help encourage people to simply type their queries in a more conversational style, using antecedents in a way they don’t now.
Typing? No “Conversational” Mode That Way
Maybe that will come, but currently, conversational search does not work if you type. In fact, that’s an excellent way to see the immense difference that conversational search provides.
Try that series of Barack Obama-related queries I did above by typing them in, rather than speaking, and it falls apart when you ask, “how tall is he,” as you can see:
Out of conversational search “mode,” because you’re typing, Google makes no attempt to figure out who the “he” is that you were referring to, so answers about Tom Cruise and Johnny Depp come up instead.
Google’s Search App For Mobile Also Has It
While all this is big news for desktop search, conversational search has already been integrated into the Google Search App for for some time.
I did a double-check on my Nexus 10 Android tablet and my iPhone 5. With the Nexus, I was able to go three deep into the Barack Obama conversation above. On the iPhone, I couldn’t get past the second question to jump to information about Michelle Obama. But that’s likely an odd glitch, perhaps a network connection, rather than any lack of the iPhone from being able to do this.
Postscript (4pm ET): Chrome for Android has voice search but not read-back built in. That appears to be staying the same. But Chrome for iOS will be gaining voice search with read-back, Google has posted. On both platforms you have full conversational search through the Google Search App.
Try It Yourself
As I said, Google hasn’t posted officially about the availability of conversational search being out there, but it is live. Android Central noticed that it’s now out if you have the latest version of Chrome. So to enable it, update. You do that by going to Settings in Chrome, then selecting Help, then it should check if it’s up-to-date and prompt if not. You’ll also have to later confirm that you want to allow voice searching to work.
On my Mac and Windows 8 computers, after updating, I was at version 27.0.1453.93. Oddly, on my Chromebook, I simply couldn’t get it to update out of version 26..
FYI, this is not Google Now for Chrome. It might look like that, and Google Now for Chrome and/or the desktop is expected, but this doesn’t provide any of the predictive search answers that Google Now provides.
Enjoy playing with it. For search marketers, it’s a glimpse into an entire new world where people might be seemingly finding you for very generic terms, because the queries no longer express all that was actually searched for.
Ironically, the much-hated “Dark Google” world over the past year or so, where Google has been withholding search terms done by logged-in searchers (except from advertisers) may have been good training on how to cope with what now will be withheld by searchers themselves — exact terms, no longer necessary as they “converse” with Google.
For searchers, it’s an advancement that may fundamentally change their search habits, where they get even more comfortable just talking to that “best friend” they’ve already been turning to for ages for answers on all types of topics.
For Google’s competitors, it’s another challenge in keeping up. Google Now is already doing amazing things anticipating searches. Now Google’s conversational search is allowing searchers to get casual with the service, plus it sets things up to entice searchers to share more of their own information directly using Google’s other services, so they can speak things like “what’s happening today” and very personal answers.
Postscript (3pm ET): Google still hasn’t done a formal blog post but has officially confirmed what we can see, that conversational search is live. The company told us:
Conversational search has started rolling out in the latest version of Chrome. You can just click the mic in the search box, ask your question in a natural way, and get spoken answers. (You’ll need to update your Chrome browser to get the new functionality.)
Google said the “hotwording” voice activation feature will be coming later but gave no specific time. There’s no word on Google Now coming to the desktop.
- Powerset Launches “Understanding Engine” For Wikipedia Content
- Google Voice Search & Search By Image Comes To Desktops
- Dark Google: One Year Since Search Terms Went “Not Provided”
- How To Turn (Not Provided) Into Useful, Actionable Data
- Google Launches Knowledge Graph To Provide Answers, Not Just Links
- Gmail Search Field Trial Adds Calendar Results To Google Search
- New Google Now Cards Include Voice-Activated Reminders
- The Amazing “Google Now” — When Google Searches Before You Think To
- Google Glass Diary, Part 3: How Search Works On Google Glass