Here is an example of searching for [greece] and hovering my mouse cursor over the related search phrase [santorini greece]:

You can click on the image for a larger preview.

Google is rolling this new feature out over the “next few weeks.”

Related Stories:

• Google Images Carousel For Tablets & Better Movie Results For Phones
• Improved Snippets, Rank Boost For “Official” Pages Among 10 New Google Algorithm Changes
• Google Upgrades Image Search: New Recent Image Filter & Old Filters Return
• Up Close With Google’s Search By Image: Hits, Misses & More
• Google Images Now Showing Fresh Images Within Hours
• Google Images Adds Subject Sorting Option

Tablet Image Carousel:

When you use Google Images on tablet devices including iOS or Android tablet’s browsers, and then click on an image, you will see the new image carousel feature. It is very similar to the smart phone experience from April, where you can swipe through images or click the arrows through images. It is also similar to the feature on the Google iPad app.

Here is a video demo of it in action:

Greg Sterling covered this at our sister site, Marketing Land, but it deserves a mention here as well.

Movie Results on iPhone & Android:

Now when you search for movie related keywords on your iPhone or Android phone, you’ll see a new swipeable ribbon at the top. It allows you to swipe the movie covers right and left and then click on them to view more details about the movie. You can see this yourself by searching for [movies] or your favorite theater name.

Here is a picture:

Then clicking on the movie shows you this level of detail and even allows you to buy the tickets:

Snippets & Page Titles

Below are changes related to snippets — the descriptions that Google shows for web pages — and page titles. I’ve quoted from the Google blog post, where that’s appropriate and paraphrased in other cases to make things clearer.

Improved Snippets: Google says that it has a change that allows it to pick more snippet text from the main content on a web page, rather than selecting from headers or menus.

Rich Snippets, More Often: “We recently announced rich snippets for applications. This enables people who are searching for software applications to see details, like cost and user reviews, within their search results. This change extends the coverage of application rich snippets, so they will be available more often.”

Better Page Titles By Using Anchor Text Less: “We look at a number of signals when generating a page’s title. One signal is the anchor text in links pointing to the page. We found that boilerplate links with duplicated anchor text are not as relevant, so we are putting less emphasis on these.”

More About Snippets & Titles

If you’re confused about why Google doesn’t just use the HTML title tag as a “signal” for creating titles and be done with it, you’re not alone. Google does use the title tag, but it may override this if it thinks it can make a better title on its own. The articles below have more about this:

• Is Google Ignoring The HTML Title Tag More Often?
• Writing HTML Title Tags For Humans, Google & Bing

Ranking Changes

Four of the changes are relating to how the search results are actually ranked:

Better “Official” Page Detection & Boosting: “We try hard to give our users the most relevant and authoritative results. With this change, we adjusted how we attempt to determine which pages are official. This will tend to rank official websites even higher in our ranking.”

Image Search Loses A Ranking Signal: “We decided to retire a signal in Image Search related to images that had references from multiple documents on the web.”

Better Date-Based Search Results: “We changed how we handle result freshness for queries where a user has chosen a specific date range. This helps ensure that users get the results that are most relevant for the date range that they specify.”

Fresher Results: “As we announced just over a week ago, we’ve made a significant improvement to how we rank fresh content. This change impacts roughly 35 percent of total searches (around 6-10% of search results to a noticeable degree) and better determines the appropriate level of freshness for a given query.”

More About Ranking Changes

The freshness ranking changes, we’ve covered in more depth recently:

• Google Search Algorithm Change For Freshness To Impact 35% Of Searches; Twitter Firehose Remains Off
• Brands & News Sites Among Winners From Google’s Freshness Algo Update, Report Says

As for ranking in general, Google uses thousands of signals to help rank its search results. If you’re trying to understand those more, please see our resources below:

• What Is SEO / Search Engine Optimization?
• Periodic Table Of SEO Ranking Factors
• Search Engine Land’s Guide To SEO

Also see our the Google SEO section of the Search Engine Land Library for many more articles on this topic.

Matt Cutts, Distinguished Engineer

The post, which was also on the main Google blog, was from Matt Cutts, who many know as the senior software engineer who oversees Google’s spam policing efforts. But if you look closely, you’ll see that Cutts is listed as “Distinguished Engineer.” That’s a title change — congrats on the promotion, Matt.

I was able to replicate the problem images by going to Google, searching for [gmail], clicking on the “more” option and selecting “Photos,” as you can see below:

After you click photos, you are presented with many images of women that are not fully pornographic, but most likely not safe for work. Here is an uncensored image:

The “Photos” option pulls back images from the Google Picasa photo sharing service, as opposed to the more prominent “Images” option, that finds images from all over the web,

My Google Image search settings are set to block images like this, but I guess those image preferences do not transfer over a Google “Photo” search.

To be honest, I never used the “Photo” option from the Google menu. I am not sure if it is even new, and I have an email out to Google to find out more information about that.

I wrote about this at the Search Engine Roundtable as Google Images possibly improving their image source detection algorithm, but it is more.

A Google spokesperson they have launched a new update for Google Images that improved relevance and quality. Google told me:

We’ve recently launched an update to the algorithm that looks at the relevance and quality of both the webpage and the image to surface more relevant results in Google Images. Improving the quality of results in Google Images is very important to us, and we’re always working hard to improve our algorithms.

Google Images has been plagued with image owners complaining about their images were being outranked by other sites who have stolen their images, with our without credit. It seems that this has now been improved.

Related Stories:

• Google Images Now Showing Fresh Images Within Hours
• Google Upgrades Image Search: New Recent Image Filter & Old Filters Return
• Google Gives You More Images, If You Ask Nicely
• Google Images Adds Subject Sorting Option

Let’s start with the new filter: Among the image search filters in the left-side column is a new “Past week” option that surfaces recent photos added to Google’s image search index. You can see it at the bottom of this screenshot:

Meanwhile, up above that filter are options to filter image by Type (face, photo, clip art or line drawing) and a couple Size options — “Larger than…” and “Exactly…” — that let users specify pixel dimensions as part of the search query. The two Size options disappeared from the left-side column when Google recently began its site redesign, and could only be found from the Advanced Image Search page. I’m pretty sure that was also the case with the Type options.

Google has been focusing on fresh images in recent months. Back in December, the company confirmed that it was updating the image search index more quickly. In April, Google began adding time stamps to the fresh images in its search results.

But if you upload the right photo (or wrong photo, as the case may be) of Michelle Obama into Google’s new Search By Image tool, that disturbing image shows up front and center among Google’s search results.

(Note: We’ve blocked the offending image so as to not give its creators the pleasure of more exposure.)

What’s happening here? A Google spokesperson talked to us about this specific search result, and we learned a couple things:

1. As Google first mentioned at Tuesday’s “Inside Search” event, the company isn’t using facial recognition in the Search By Image tool.

2. The Search By Image algorithm is looking at other things in the uploaded photo — the general shapes and proportions of what’s shown in the image, the colors, the outlines of Obama’s hair and clothing, and so forth.

It makes sense: Because the creators of the offensive image only changed her facial features from the image that I uploaded, and because Google isn’t using facial recognition, the Search By Image algorithm is actually making what I’d call the correct match. Without facial recognition, the disturbing image is essentially the same as the real image that I uploaded.

Will Search History Be Repeated?

Back in 2009, Google initially removed the offending image because it was hosted on a site that was serving malware, which is against Google’s terms of service. But the image showed up on other, “clean” sites and quickly made its way back into the image search results. That prompted Google to buy ads against the search term “michelle obama” to explain to users why search results are sometimes offensive.

Speaking about the Search By Image case above, Google wouldn’t speculate if they’d run similar ads now. The spokesperson said that Google views these as two very different cases: In 2009, an offensive image was being shown for a very broad text query (“michelle obama”). In this case, an offensive image is being shown in response to a very specific, user-uploaded image — an image that is closely associated to the offensive version.

In fact, the offensive image matches the original in every way but one: the First Lady’s face. And without using facial recognition, there’s no way for Google to distinguish between the two.

One Final Note

Back to that first paragraph: If you’re wondering why the offensive image doesn’t show up anymore on searches for “michelle obama,” Google says it’s because of algorithmic improvements, not any specific filtering on her name. The spokesperson said that the company’s internal metrics show that they’re doing a much better job of identifying the authoritativeness of individual images — and the offensive image is not authoritative for Michelle Obama’s name.

P.S. See Should Rick Santorum’s “Google Problem” Be Fixed? for a look at how this case compared to Rick Santorum’s “Google Problem.”

Search By Image isn’t the first start-with-an-image search engine, and isn’t even Google’s first version of this kind of search tool. (TinEye has offered reverse image search since May 2008, and Google Goggles was Google’s first search-by-image tool back in 2009.) But Google is the big and new kid on the block, so we’ve taken an in-depth look at how Search By Image works — including when it’s at its best and worst.

How Google Search By Image Works

Search By Image is available now at images.google.com or via the “Images” tab in the left-side menu on Google.com. You should see a small camera icon on the far right side of the search bar.

There are several ways to access it:

• Drag and drop an image on the search bar
• Click the camera icon to upload an image from your computer
• Paste the URL of a photo on the web into the search bar
• Use the Chrome or Firefox extensions that add a search option to your computer’s contextual menu (right click)

After you supply a photo, Google goes to work. The company’s blog post explains that Google analyzes the image, creating a mathematical model based on shapes, lines, proportions, colors and other elements. It then matches the model against images already in Google’s index. Google then does page analysis to take a text-based guess at what the image is, which is part of the process of identifying the image and returning similar results.

Search By Image looks for similar content on the web, so unique or never-before-seen images won’t work well.

No Facial Recognition

One thing that Google emphasized Tuesday is that Search By Image does not use facial recognition. Although Google’s Picasa photo service does facial recognition and others like Exalead do, too, it’s become a hot-button issue on the privacy front. Facebook’s recent announcement that it would use facial recognition to identify friends in newly uploaded photos is the subject of government investigations, and Google has gone to great lengths to deny any plans of using the technology.

So, when you upload a mug shot, Google’s Search By Image results are interesting:

When I upload a photo of myself, Google is able to find web pages with the same image (1), even some that are cropped down the middle. But without using facial recognition, Google only shows similar images (2) that have a similar combination of shape, proportion and colors to my original. (Should I be bothered that a stuffed monkey shows up on row two as visually similar to me? Thanks for nothing, Google.)

By the way, the filename doesn’t appear to matter when you upload an image. I uploaded that mugshot of myself as yDSC_6801.jpg and as matt-mcgee.jpg, and both searches returned the exact same results.

But the browser you use might make a difference. The screenshot above is from Firefox — for some reason, the “visually similar images” returned were slightly different when I uploaded the same mugshot in Safari. Have a look:

See also our follow-up story,Disturbing Michelle Obama Image Makes A Case For Facial Recognition In Google’s New Image Search.

Search By Image: The Hits & Misses

If there’s no facial recognition, identifying people may not be the most recommended way to use Search By Image. So, how is this new feature supposed to be used? In the intro video, Google suggests that it’s for identifying “places, art and even mysterious creatures.”

So, let’s try a few sample images and see how well Google does, shall we?

Identifying Places

Death Valley, California, is one of my favorite places on Earth. (You have to visit it if you’ve never been there.) Zabriskie Point is one of the most well-known and photographed spots in Death Valley. I expected Search By Image to recognize one of my photos of it, and Google didn’t disappoint.

I thought that the Rock and Roll Hall of Fame and Museum in Cleveland might be a little harder to match, but Search By Image nailed it.

So, let’s make this more difficult. Another favorite place is the Pro Football Hall of Fame in Canton, Ohio. You’ve probably seen it from the outside — it has a half-dome above the entryway that almost looks like a church building. When I visited there last summer, I took a photo looking up at the inside of the half-dome. Search By Image was a complete miss on this one.

Notice the text above my uploaded photo? It’s asking me to help by providing a description. When I add “football hall of fame” to the search box and try again, Search By Image does better. It finds the official website and the (obligatory) Wikipedia page, and also locates a couple similar images (one of which is my photo uploaded to Flickr a year ago).

Still, that seems like “cheating” when you have to add some text to get the image matching correct, doesn’t it? Let’s try a couple other kinds of images.

Identifying Flowers

I have a thing for flower photos. Problem is, I can’t identify about 90% of what I shoot. Like this yellow flower that I’m sure everyone reading this can identify … but can Google?

Nope. I’m not the only one who doesn’t know what it is. (If you know, leave me a comment, please?)

Another miss: I was surprised to see that Search By Image couldn’t identify this photo of a sunflower.

Search By Image got the color right, and recognized the circular shape inside the photo. But I’m guessing the original photo was too macro to identify as a sunflower; something showing the entire flower might’ve worked better. (But I don’t have any such photos.)

Two Final Challenges

I grabbed two images from my collection that I thought would be real challenges for Search By Image. The first is a photo I took at Disney’s California Adventure — it’s Roz, the hysterical character from Monsters Inc. She’s the very definition of a “mysterious creature,” which is how Google suggests I use Search By Image! Sure, I know there’s no facial recognition, but in her case … might that actually help? Not really.

Apologies to all of the wrestlers and everyone else that was identified as “visually similar” to Roz. But that’s Google, not me. You can reach them at 1600 Amphitheatre Parkway, Mountain View, CA 94043.

My last image is a fairly nondescript photo of the Seattle skyline that I took from atop the Space Needle. Since the very distinctive and oft-photographed Space Needle wasn’t in the photo, I expected this might be tough to distinguish from any other city skyline. Boy, was I wrong.

That’s pretty impressive to me: Best guess for this image: seattle skyline? Yep. Exactly. Well played, Search By Image. I’m guessing it helps that a lot of people have taken similar images from atop the Space Needle.

What About TinEye?

TinEye has been offering similar image search capabilities for a couple years and boasts that it has an index of about two billion photos right now. One big difference is that TinEye doesn’t provide text or search results that tell you what you’re looking at; it only returns similar images.

But how well does it do that compared to Google?

I uploaded each of the same images into TinEye to see how well it did and here — minus what would be an overload of screenshots — are the results:

• Matt McGee mugshot – TinEye correctly identified me by name and displayed several exact matching images
• Zabriskie Point – correctly returned a set of matching images
• Rock and Roll Hall of Fame and Museum – TinEye couldn’t find any similar/matching images
• Pro Football Hall of Fame interior – no similar/matching images
• mystery yellow flower – no similar/matching images
• sunflower macro – no similar/matching images
• Roz – no similar/matching images
• Seattle skyline – correctly identified and displayed similar/matching images

Final Thoughts

Both Google’s Search By Image and TinEye performed very similarly with the images that I uploaded from my collection: They did very well for obvious landmarks and things that are commonly photographed and found online, and not very well for other types of images. The Rock Hall building is the only image that Google identified and TinEye didn’t.

But my sample was extremely limited and designed just to get a general sense of how Google’s new feature works, and how it compares to one that’s been available for years now. To get a more accurate sense of the two, you’d need to do probably dozens of more uploads and more detailed testing on whether things like image size matter (my colleague Gary Price thinks it does, based on some of his own tests with Search By Image).

For me, the final verdict is this: If you need to know what an image is and plan to use Google’s Search By Image or TinEye, you better hope there are already a lot of similar images online and, in Google’s case, a lot of web pages (preferably one from Wikipedia) that describe the image.

Postscript: See our follow-up post, Disturbing Michelle Obama Image Makes A Case For Facial Recognition In Google’s New Image Search.

With Voice Search for the desktop, which is a feature of the Chrome browser, users click a microphone icon in the search box, and then speak into their computer’s microphone.

Part of the idea, according to Google execs speaking at a media event in San Francisco, is to make voice search ubiquitous and increase awareness, to increase adoption. Already, mobile speech inputs to Google are up six times since being introduced.

Besides “training” users to begin to search by voice, Google says it anticipates the new feature will be used by folks with hard-to-spell or complicated searches, or by those who literally have their hands full.

This new interface will be rolled out to Chrome users on Google.com in English this week.

Google Search by Image for the desktop, part of Google Image Search, builds upon the image-recognition technology the company previously rolled out for mobile in Google Goggles. As in Goggles, users input an image, and Google returns results it believes are related to that image, along with similar images. Users select an image by dragging and dropping, uploading, or inputting an image URL. Installing a Chrome or Firefox extension enables searching by image by right clicking on the image on the Web.

The company demonstrated a few ways people might use desktop search by Image. In one, someone had forgotten exactly where he’d taken a particular picture on vacation. When the photo was dragged and dropped into the search box, Google returned results related to a particular Greek island. The feature works best for images of things that are quite well documented, such as often-photographed tourist attractions or images that are seen frequently online.

Image searches are treated like any other query in terms of privacy and data storage. Google search by image is rolling out today globally.

For example, a search for [flowers] returns these results and when you hover over an image, it expands:

But when you search for [flower images] you many many more images:

You can trigger the web results to show more images by adding keywords like “photos”, “pictures”, and “images” to a query.

Related Articles:

• Google Images Adds Subject Sorting Option
• Google Launches Instant Previews
• Google Images Now Showing Fresh Images Within Hours
• Google Image Search Adds SVG Filter & Google Displays Satellite & Terrain Maps
• Find Similar Images Now In Google Image Search