Google multisearch – search by image and text at the same time

Google multisearch is Google’s latest innovative search feature that let’s you search by image and then add text to that specific image search. Google says this lets searchers “go beyond the search box and ask questions about what you see.”

What is Google multisearch. Google multisearch lets you use your camera’s phone to search by an image, powered by Google Lens, and then add an additional text query on top of the image search. Google will then use both the image and the text query to show you visual search results.

How Google multisearch works. Open the Google app on Android or iOS, click on the Google Lens camera icon on the right side of the search box. Then point the camera at something nearby or use a photo in your camera or even take a picture of something on your screen. Then you swipe up on the results to bring it up, and tap the “+ Add to your search” button. In this box you can add text to your photo query.

Here is a GIF of this in action but you should be able to try it yourself in English, in the United States:

Here is a static image of the flow of how this works:

How is Google multisearch helpful. Google said this feature can help you narrow down your searches, here are some examples of how multisearch can be helpful.

• Screenshot a stylish orange dress and add the query “green” to find it in another color
• Snap a photo of your dining set and add the query “coffee table” to find a matching table  
• Take a picture of your rosemary plant and add the query “care instructions”

MUM not yet in multisearch. Google made a comment in its blog post saying “this is made possible by our latest advancements in artificial intelligence, which is making it easier to understand the world around you in more natural and intuitive ways. We’re also exploring ways in which this feature might be enhanced by MUM– our latest AI model in Search– to improve results for all the questions you could imagine asking.”

I asked Google if Google multisearch currently uses MUM and Google said no. For more on where Google uses MUM see our story on how Google uses artificial intelligence in search.

Available in US/English. This feature is live now for me, and should be available as a “beta feature in English in the U.S.” Google said. Google also recommended you try it with shopping searches.

Why we care. As Google releases new ways for consumers to search, your customers may access your content on your website in new ways as well. How consumers access your content, be it desktop search, mobile search, voice search, image search and now multisearch – may matter to you in terms of how likely that customer might convert, where the searcher is in their buying cycle and more.

About The Author

Barry Schwartz

Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on Twitter here.