Bing’s Stefan Weitz: Rethinking The Search Experience

Way back when, in the dying weeks of 2009, I asked the question, “Where does search go from here?” It seems that everyone agrees we’ve barely scratched the potential that is web search, but what might that scratch reveal? What will our searching look like in two years? In five years, or even in ten years?

I think, for simple keyword contained needs, search as we know it works fairly well. It’s where search bumps up against the complexities of human behavior that it tends to fall short. What if we don’t know what we’re looking for? What if the search algorithm gets tangled up in the ambiguities of language? And what if we aren’t on a single fact finding mission, but are, in fact, engaged in a much longer quest for information. Can search be the guide that helps lead us through complex decisions? The answer today, I think most would agree, is no, not really. Sometimes, many times in fact, web search as we know it just isn’t that useful.

For the next few columns, I want to explore where search might go, not from a technical perspective, but from the user’s perspective. How will we use it? When will we use it? What will make us come back and use it again? And why?

Enter the “Decision Engine”

These are the questions Microsoft decided to tackle with the introduction of Bing. I had the chance to chat with Stefan Weitz. Stefan’s title (Director at Bing Search) doesn’t really explain very well what he does. Think of Stefan as a bridge between the technical wheels of Microsoft’s search machine and the real people that have to use it. Stefan is the eyes, ears and often the mouth of Bing in the real world.

He’s had the job for a little over a year now and, by his own words, it’s going “shockingly well.” Anytime you sign up to be the evangelist for Microsoft’s search efforts, you can expect more than a little cynicism might come your way. Weitz knew this when he signed up for the gig, but “it’s a lot more positive than most of us expected it to be, to be blunt.” Microsoft has signaled that they are serious about search and Bing has proven to be an entry that has to be taken seriously.

What could search do better?

Stefan and I touched on a number of topics. When you talk search with Stefan, it’s hard not to stray off into fantastic images of what search could be. While these are inevitably fascinating, in today’s column, I want to restrict the scope to a monumental decision made by Microsoft about how they’d compete in the search arena. Why a “Decision Engine?” Why did they break off from the Google path of algorithmic relevancy. As Stefan explained, it has a lot to do with the pieces of search that still don’t work that well.

Stefan Weitz: This idea of a universal notion of relevancy worked really well in the earlier days of the web. We had a smaller web, we had a more static web and we had a web that really was a web in the true sense of that term. That was the way PageRank was constructed and that algorithm was quite brilliant. I still think that it’s quite brilliant. But it does seem a bit chaotic that we’re using that same notion. If I’m looking for the best hospital to treat cancer, just think about how ridiculous that model actually is. We’re now relying on popularity and in links to determine that? That doesn’t make any sense.

Relevancy is relative. It is about the intent of the user, first of all. What is the user trying to do? Then, secondly, what do you know about the user or the query that could help to better refine the results? Again, don’t think about results as more links. Result could be an answer. The weather in Seattle is a fairly canonical answer. Relevancy could be canonical. Relevancy could be definitive, where we know 99% of people ultimately go for this type of answer.

The semantic challenge

So, relevancy doesn’t always provide the right answer, or, more correctly, the most useful answer. And part of the problem comes with a decidedly human factor that doesn’t play well with artificial intelligence—language. Language, as any linguist will tell you, comes wrapped in nuance and ambiguity. Language depends heavily on emotional heuristic processing. That’s why we communicate so much more effectively when we’re face to face with someone. Imagine then the challenge that faces a search engine when accurate determination of intent depends on automated parsing of language.

Weitz: There is this more semantic model of the web, and it’s not semantic search per se, but it is this notion that engines should be more human—engines will begin to understand the information they are crawling. Here’s an example. If I walk into a conference room of people and ask why are there no jaguars in this conference room everyone is going to look at me like I have two heads. They would say “why would there be a jaguar in this conference room?” Jaguars are mammals, they are carnivorous. They live outside. Or… Jaguars are cars. In any case, it makes no sense to have a jaguar in a conference room. However, if you go to Google, or Bing, or Yahoo and type in “jaguar” and “conference room” you’ll get back in excess of 5 million hits.

That’s one of those big investment areas we’re making: How do we help the engines create this ontology or this model. As they are crawling this mass of data, how they can construct that index in a way where they understand the context of the data. So they can receive a query like “jaguar” and “conference room” and begin to start refining the question or throw out some different options and once they understand what it is that I’m trying to ask, then they can do a better job retrieving the information.

Understanding data

So, the first challenge facing the Bing team is nothing less than creating an algorithm that employs “common sense.” The second is trying to match a mass of data that sits in it’s index to the intent of the user in a useful way.

Weitz: Once we understand it, it’s not just about throwing a bunch of docs that may have a high PageRank back at you, but this is about the model the web is moving towards: it’s much more dynamic, much more social, much more “real time,” much less static. One of the things we do, where it makes sense, is to pre-process the billions of data points that we have from all different data sources and send the query a response that makes sense.

It doesn’t have to be just another link. For example, it could be Farecast, where we see the prices going up and down for different airlines. It could be opinion ranking, where we actually look at all the reviews of restaurants across the web and summarize those in a smart way. We can throw back more than just an opportunity for more exploration, to be nice about it, and we can actually contribute to the user some knowledge which would have taken them hours, or hundreds of hours or an infinite amount of time to tackle it on their own.

Why a “Decision Engine?”

Say what you will about Bing, they have not shirked from the challenge that faces web search. They’ve picked two gargantuan nuts to crack. Does rising to such a challenge make business sense? What was the reasoning behind calling Bing “The Decision Engine?”

Weitz:I’ll give you two answers on that one. The first, from a pure business and marketing standpoint, we had a lot of features and value propositions but they don’t fire as frequently as they need to really differentiate this. When you talk to users and ask them, “Are you happy with your search engine?” most people say “I’m delighted, I’m happy, I don’t need anything more.” So early on we were thinking: How do we position this product? How do we capture mindshare? How do we get some people to try it and see if they like it?

If we just came out and said, Microsoft has a new search engine, because people are generally happy with search engines, there’s just no real reason to give it a shot. We need to do something that is “of the category” but is it’s own category. That was the rationale behind the positioning of “The Decision Engine.”

But, of course, that has to be backed up by a product. All of Bing started with us getting back to the data, trying to get back to what are the real problems with search today. There’s a ton of different ones we could talk about, but in this particular context, the Decision Engine, the two biggest problems that stood out to me were the fact that about half the time people spent searching on the web are in sessions that span more than half an hour. What that really means is that is you spend 10 hours a month searching, 5 of those hours are going to be spent in very quick, under 3 minute, in-and-out sessions. But the other 5 of those hours are going to be made up a few long sessions. As we dug into that behavior we found that people were using engines that were designed for very atomic queries—a query, keyword, out model—to conduct these longer sessions. There was a mismatch in usage and utility.

The second thing we saw was that people were using these engines to do more complex tasks. When you ask people where do they go to book travel or research travel the number one place was still one of the aggregators: Expedia, Travelocity or Orbitz. But the number two place, and it’s not far behind number one, is a general purpose search engine. Think of the futility they must be feeling when they are using an engine that’s looking at keywords and links and all of that when they’re trying to book travel. People’s expectations of what an engine should do are changing and they’re using them as a starting point for a lot of their tasks. Those were two data points that can inform how we build this new product.

The Decision Engine was built around three big areas. The first was providing great core results. That’s the standard “block and tackle,” keyword to keyword algo based search. That’s what search is used for a lot today so we had to nail that. We spent a lot of time on that and that’s what you’re seeing when you do the head-to-head comparisons with Google. But that’s still a search engine. The Decision Engine comes when you add in the other two big things.

The first is that organization of results to help people explore topics that they don’t understand. Can we do a better job with related searches? Can we organize results using categorized search. Can we semantically break down the 160 million results in a way that makes more sense. The third thing is how can we provide tools that help you make decisions? We focused in the initial release on the travel, the shopping, the local, the health. We built fairly complex computer science tools to help you when you do decide you want to a search engine and book that trip to Florida. What can we do differently that will help you get that job done faster? In the travel vertical, it’s the Farecast technology, the ranking within airfares… all those types of things. That’s how it practically manifests in the engine and it is designed to respond to those data points I mentioned earlier.

Is Bing suffering from indecision?

I have no quibbles with Microsoft’s Decision Engine strategy. I actually agree with the reasoning behind it. My point of contention, which I talked about in my last column, was that I don’t see the differentiated experience Bing promises in enough searches. I mentioned this point to Stefan:

Weitz: Your point is valid, in that I don’t see it (the Bing difference) enough. We know that. We get that as well. What you’re starting to see, especially in the release we did towards the end of 2009, is a lot more “platformizing” of search. A lot of the functionality we’re delivering, the categorization, the definitive cards that show you the best match for that site, the new thing we have called Task Pages—the best examples now are either weather or events—those things are being built as a platform so we as we get smarter on crawling, as we get smarter on indexing, as we do more deals to ingest quality proprietary content we can begin to fire those types of experiences more often.

Bing and its market share aspirations

A consistent theme ran through most of Stefan’s content, a positioning of Bing against the 800 pound gorilla of search—Google. I asked Stefan what Microsoft’s goal were as far as gaining share from Google:

Weitz: Competing with Google in classical ranked algo search isn’t a thing we spend tons and tons of time thinking about. We obviously try to understand what they’re doing so we can match or better that.

Really, the question about how we compete with the traditional engine assumes some type of zero-sum game. We think by enabling all these other decision type scenarios, we’re actually increasing the overall pie. We know you’re using search in all these non-traditional ways. What can we do to help accommodate that? It might not be something you would think about in a traditional search. For example, we just launched the education Task Page. When you do a search for the University of Washington, we’ll go out and assemble a bunch of information from across the web. We go out and pull all this data into a very clean results page that looks nothing like the typical search results page. It’s almost like a programmed editorial page.

If you want to get into the competitive aspects, the way we want to compete is to help people realize they should demand more from search engines. They should demand more from this technology. They should expect to see the user experiences and user interfaces which better match what they’re trying to do. That’s where we spend a lot of our time—on that visualization of the data.

Today’s engines, in most cases, still put too much of a burden on the user to sort through the 161 million links for cat. That’s silly. It’s not logical. There are many ways that we think are superior to try to capture intent and display that knowledge back in a way that will cause the user to say, “Wow, what I thought was the pinnacle of search, the state of the art, which was a very fast URL ranked result based on a keyword, that is literally the 1.0.”

We will continue to introduce these verticals, in pretty short order, frankly. The sum of those parts will become a very differentiated experience that will expand how people think about search. Competitively, that means that when they try that type of search on other engines, they won’t be as satisfied and they’ll come to Bing more often.

Stay tuned…

In the next column, Stefan and I will talk more about the potential of search, and Bing in particular, as we look beyond the immediate horizon. Next time, I return to the question I first asked: Where does search go from here?