Google Paper: Better Image Search Though VisualRank / Image Rank

A Google Prototype for a Precision Image Search from the New York Times
covers a new research
paper (PDF format) from Google that talks about a way of ranking images based
on analyzing "visual links" between them.

Image search at the major search engines today largely relies on looking at
words that are used around images — on the pages that host them, in image file
names, and in ALT text associated with them. No real image recognition is done by
any of the majors. Search for "apples," and they haven’t actually somehow
scanned the images itself to "see" if they contain pictures of apples.

The method in Google’s paper changes that. In short, a group of images
retrieved for a query using traditional search methods is then further analyzed.
Image recognition software finds which images in the group seem most similar to
each other. It then estimates "visual hyperlinks" between them to produce a
final ranking.

The last part is important. No actual hyperlinks on the web are used to rank
the images, if I understand the paper correctly, other than in the first
traditional retrieval process. Instead, the algorithm guesses at how the images
would be linked together, with those being most similar having more virtual
links to each other. As a result, the most "linked to" images are calculated to
rank first.

The image above comes from the paper and shows examples of images found in a
search for [mona lisa]. The lines illustrate how they are all estimated to link
together, with the two in the middle (as shown in the close-up below) deemed the
most relevant based on linkage:

The New York Times article says the researchers call the method "VisualRank,"
though that term is not used in the actual paper, which is entitled "PageRank for
Product Image Search," coming from how the method was applied to product search
results as a test. The paper itself talks of Image Rank at one point, so
VisualRank might be a new name the researchers are trying out.

Image recognition isn’t new or unique to Google, though this twist on using
virtual hyperlinks is. For background on what others are doing, see
Teaching Google To See
Images from Chris Sherman last year. It covers players such as Riya. My
article on Polar Rose,
Polar Rose Promising Face Recognition Image Search, also provides some
further background on image recognition as well as the
Google Images Labeler that
relies on human judgment to identify image.

For further discussion,
see Techmeme.