Bing 2.0 “Visual Search” Launches, Allows Search By Pictures

Microsoft has unveiled Bing 2.0 at TechCrunch50 today, also known as Bing Visual Search, which allows people to “search by picture.” After much speculation following Microsoft’s annual meeting last week in Seattle, many had expected to see an update to Bing Maps,  integrating  Silverlight technology to display images of local businesses and locations within map search results. We got a sneak preview of Visual Search in August, we can confirm this is the first in a series of tweaks and other major features set to roll out through the end of the year. Here’s a closer look at today’s launch.

What is Bing Visual Search?

Bing Visual Search lets searchers browse easily through a slick interface of “structured data sets from trusted partners” using Sliverlight technology. At launch, Bing Visual Search will earn a spot on the homepage search categories, just under Travel, although depending on the homepage image of the day, those links can sometimes get lost in the background colors of the photo. Like Travel, Visual Search is also missing from the top navigation once into Bing search results. For general web search queries where there is an applicable Visual Search data set, a teaser will appear within the results to explore related data visually.

Bing Visual Search

The concept behind Visual Search is simple: use clear imagery to help users sort through large sets of data easily. Certain categories of search lend themselves more easily to this than others, likely the reason why Bing has launched this feature in beta with a fairly limited set of visual information: cars, animals, people and products. Users must have Silverlight installed on their browsers to fully experience Visual Search. According to Rich Internet Application Statistics, nearly 73% of users do, and Silverlight adoption has been growing fairly quickly since relied upon Silverlight for live streaming and on-demand video coverage during Beijing 2008, and will likely use it for the upcoming Vancouver 2010 games.

Clearly, items in the merchandise category do have a natural fit with Bing Shopping – users can easily identify differences between digital cameras or handbags at the initial search, and then move on to the comparison stage. Other categories capitalize on the popularity surrounding common searches on celebrities and entertainment, namely popular music, books and movies, and typically can funnel users to shopping results and categorized search at the next level. Research-based topics including politicians, US States and items like the periodic table are useful applications, but perhaps only to a limited audience, such as younger students working on school products. However, this also affords Bing the opportunity to appeal to a new generation of searchers, who are highly dependent on visual cues and ease of use, as iPods and iPhones have shown us. Of course, visualization does have additional appeal to the middle demographic using those products as well.

App-Store-Ex-on-BingDespite Steve Ballmer’s reported hatred of iPhones, one of the most interesting Visual Search data sets includes Top iPhone Apps, where the icons are sortable by popularity, price, release date, name, or category  (such as games or social networking). If you sort the apps by popularity and select the top result (currently “AppBox Pro by e2ndesign”), you get something akin to a instant answer-style result (example to the right), which immediately opens iTunes. Typically, this prominent placement might indicate an official iTunes affiliate relationship, where the publisher (Bing) would be earning a % of revenue on every app then purchased and downloaded from that click event. At launch, there no  such deal in place, Bing assures us. However, not every web search result for iPhone apps returns this instant answer format, so the criteria for instant answer display may be something other than an affiliate deal.

Features of Bing Visual Search

Bing Visual Search is understandable and intuitive. However, the initial galleries in this beta phase are primarily novelty-type searches. For that reason, these starter searches are a win for Bing. Case in point: A more serious NFL fan would go straight to or a fantasy football/statistics driven-type site for in-depth statistics, but a casual fan might want to simply know the basics when looking at teams or players, which Bing Visual Search provides.

Visual Search gives you a quick, cursory look at all 32 teams and lets you sort through 2582 players by popularity. Unfortunately, Brett Favre is first, as Microsoft confirms that Bing is the determining factor for popularity, and not the data provider (though in some cases they may determine top results) but hasn’t yet confirmed the calculation method for this query. For instance, popularity could be generated by number of search queries, or by number of mentions in the search index. By either measure, it is surprising that Michael Vick does not show up in the popularity index, but does appear when searching by player name, although his profile data is incomplete. Bing says they are looking into why this may be the case.

If I want to see how my New England Patriots compare to their first opponent, the Buffalo Bills, I can do so at a glance, by rolling over the Patriots logo. I see what I need to know appear around the search box, and repeat the process for the Bills:


With the 5th ranked offense, and 10th ranked defense (not to mention 3 Superbowl wins), and a power ranking of 2, New England is the clear winner over Buffalo’s 25th ranked offense and 14th place defense, at least according to Bing.

Note: in the lower right corner of Visual Search, you’ll see the source of the data set (when available). In this case, It’s Fox Sports, an MSN partner, and STATS.

If I click through to see more information about the New England Patriots, I see essentially the same SERP from Bing 1.0, with categorized results below the top results. The notable exception is that Bing 2.0 has added social sharing tools that enable me to invite friends to watch the game with me below the instant answer and above the #1 result. No offense, I am really grateful for my nearly 1,000 followers on Twitter, but no way I am making clam chowdah and buffalo wings for all of them, nor could they all fit in my media room!


As a carryover from the initial Bing roll out, further categorized searches appear below the first set of results (which incidentally, include an unnecessary duplicate listing of the Patriots homepage, and a second, non-indented result from Wikipedia) and provide several (superfluous) results for Patriots schedule information, three-plus relevant results for merchandise, as well as three results apiece (+more option) for Rumors, Rosters and History. In Bing 2.0, the main image results move up from the bottom slot (just above video results featuring the infamous Snowbowl, among other relevant and more current results), yet they still don’t make the cut for an above the fold presence, even with my 22″ monitor and 1600×1200 resolution. And like so many others (besides Gisele Bundchen), I really want to see Tom Brady front and center when I go calling for him. Oh well, I suppose I should be satisfied that he is currently #3 in Bing’s popularity index of NFL Players in Visual Search. My only other criticism of this page might be that the plug to get Bing users interested in Visual Search happens again at the bottom of the page.

Bing delivers on travel destination decisions

As limited as the initial galleries in Bing Visual Search may be, this roll out does at least include 231 travel destinations, which in theory, should be a complementary section to Bing Travel. For now, clicking on a destination provides standard search results, and no integration exists with Bing Travel’s price predictor or other travel content other than image and video search results. The redeeming aspect is being able to sort travel destinations by popularity according to Bing and/or Fodors, the trusted data provider of this visual gallery, and this example produces some interesting results in the Top 10 travel destinations, although Las Vegas typically appears at the top of many leisure travel lists.

From a usability standpoint, I might  like to see the most popular items at the top, and the least at the bottom, but this arrangement does vary by collection and sorting criteria (even though we did see the most popular NFL player at the bottom of the screen.)  Bing likely has done some user testing on this feature, and will continue to do so and make tweaks as needed.

Top Travel Destinations According to Bing Visual Search

More impressive, is the data that appears in the search box when you roll over some of the choices available in the travel set. With Las Vegas, for example, it lets me know that I have approximately a  one hour flight time from Salt Lake City (SLC) to McCarran (LAS), seemingly basing that (very accurate) answer on IP location versus sign-in details. Several other locations I tried, such as Boston (BOS) and Portland, Maine (PWM) were spot on for flight times, even with the latter (PWM) requiring at least one stop over. But not all locations provide an approximate flight time.

Another interesting feature within the travel category, is filters by best time to visit. Selecting “February” and “North America”, I get a fairly broad list from Bing (43/231), but at this stage of the game, Bing doesn’t understand whether my intent is go skiing or lay on the beach. This layout does however, play nicely into the ‘inspiration’ angle of travel planning.


For the record, my chosen intent would have been skiing and snowboarding, so at least nine of those options are relevant results. But by this example, we can see that Bing still has ground to cover in terms of matching user intent, and more fully integrating Visual Search with other features in Bing Travel and web search results.

The end game

When we first saw Bing Visual Search in preview, it had even fewer data sets than its current collection at launch, so while this beta round is admittedly light on content, it is an interesting and creative play towards changing the expectations of searchers. However, many might argue at this point that Visual Search is simply adding an intermediate (and perhaps unnecessary) step before the most relevant results, something Bing simply cannot afford to do in the competitive landscape. They need to get to the right set of results, and quickly. Calling Visual Search a game changer may be premature, given this limited data set, but is it a step in the right direction? Possibly. In some cases, Bing is the clear winner here. For instance, viewing the Periodic Table in Visual Search is far more visually engaging and useful than an almost confusing search result in Google.

Undoubtedly, many of these novelty searches are fun and could potentially drive revenue directly to Bing, such as iPhone app search or other shopping applications. Beyond that, I would expect to see additional collections that might provide both novelty and utility, such as recipes, food and drinks, real estate (best places to live) or home and garden. And I can’t help but wonder why some  categories didn’t make the cut for the sake of utility, particularly since we are on the cusp of the holiday shopping season. You might think that toys and games (beyond electronics) or clothing/accessory categories (beyond handbags) would have made the cut. And especially since Zappos is the provider of this image set, why not shoes as well? If we aren’t seeing these categories roll out in the weeks immediately following this initial launch, Bing may have simply missed the boat on what otherwise would have been a hit with both users and retailers this holiday season.

Related Topics: Channel: Consumer | Microsoft: Bing | Microsoft: Bing Image Search | Top News


About The Author: is Director of Audience Development for Search Engine Land and Marketing Land. She is responsible for increasing readership through owned, earned and paid media channels. In addition, she assists in programming sessions at Third Door Media's Search Marketing Expo conference series and manages speaking engagements for editorial staff. Follow her on Twitter @elisabethos.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • seladmin

    Comments should be open and working now.

  • Ian Macfarlane

    Major correction on Silverlight market share stats – those stats show that nearly 73% do *NOT* have Silverlight installed.

    So only around 27% of people worldwide have Silverlight installed. Stats for the US are only about 1% more than the worldwide stats.

    Compare this to around 2/3 of people with Java installed, and over *97%* with Flash installed.

    Silverlight = not supported by most of your users.

  • jonlomb

    Microsoft obviously has an uphill climb to gain search engine market share against the Google juggernaut. It is praiseworthy of Microsoft to attempt to innovate and bring new ideas into the search engine sphere.

    I can’t help wonder if in the long term, if they will only succeed in making Bing as well as Google, a better search engine. Google is unlikely to allow Microsoft to gain much ground in terms of search engine functionality. Google will inevitably come out with their own parallel implementation of any Microsoft innovation.

    Any game of leapfrog, however, is a win, win situation for users. This competition merely serves to illustrate how fundamentally different and complementary the two companies are. At the end of the day, Microsoft, because of it’s historical dependence on it’s own proprietary software will always be vulnerable to Google’s open platform, or platform neutral stance.

    It is too early to hazard a guess at this point, whether or not either Bing or Chrome will gain any substantial market share against the overwhelming advantage their rival products currently enjoy.

    Let’s see who gains market share over whom, in the search engine and operating system marketplace over the next 10 years. That is a long term outcome that does not require a crystal ball to see clear winners.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide