Microsoft’s “Third Era” Of Search Begins With Departure Of Search Chief Christopher Payne

Microsoft’s top search executive leaving company from the Seattle PI covers Microsoft’s Christopher Payne reportedly leaving the company. The AP also reports similar news from its sources. Payne took over search efforts at Microsoft back in 2003. He oversaw what I’d call Microsoft’s "second era" of search, that of developing a crawler-based search engine to take on Google. Below, some history on the second era, plus the company’s "first era" of human-powered search and how a reorganization under the direction of Steve Berkowitz marks a new "third era" of battling for market share.

First Era: Human-Powered Search (1997-2002)

Search isn’t some relatively new effort that dates back to 2003 at Microsoft. Search, especially web search, is something the company has seriously pursued since 1997. In its first era, Microsoft started out with a crawler-based search engine, one that creates listings by using automation to harvest material from across the web. It then migrated to building a very good service that relied primarily on human power, human beings to either catalog the web or customize top search results with hand-picked answers. Bill Bliss was the person in charge during most of this period. Here’s how it unfolded, over the years:

1997: "Yukon" Search Rumors Emerge

In August 1997, rumors emerged that Microsoft was building its own search engine code named "Yukon." Microsoft went public with its plans in October 1997. Rather than build its own technology, Microsoft decided instead to lease search from Inktomi. Searchers were promised "the freshest, most current index available to consumers."

There was much expectation in some quarters that Microsoft’s entry into the space would wipe out other players, similar to how Microsoft had succeeded with software. But I thought this unlikely, given that Microsoft wasn’t doing anything special. From my story at the time:

Microsoft’s entry poses two big questions: will it make searching the web uniquely better, and related to this, will it adversely affect the major search engines? The answers seem to be no….

Microsoft is not adding any special search technology to the mix. There is no killer search app that will be created. This may occur in the future, but at launch, it will remain more of the same.

Microsoft itself wasn’t even positioning the new service, still without a formal name, as a search killer. Also from my story then:

"Our goal is not to make MSN.com the number one search site on the Internet," said Ed Graczyk, MSN’s lead product manager. "Our goal is to make MSN.com the number one site on the Internet, period. And search is a key component of that."

1998: MSN Search Debuts

Nearly a year after announcing it would get into search, Microsoft finally unveiled its own search engine dubbed MSN Search (an idea to call it "Internet Start" was dumped). As expected, it was powered by Inktomi. It wasn’t particularly groundbreaking, but Microsoft promised that it was still only the first generation of what would come in search:

"This is an evolution of what our search service is going to be," said MSN product manager Nichole Hardy. "Our ultimate vision is to make MSN.com a place for people to get things done on the web."

1999: Musical Chairs With Search Providers, MSN Search With Human Power & Bill Bliss

Microsoft made a big deal about how great Inktomi’s search technology was, the technology that powered MSN Search. But when a better business opportunity with AltaVista emerged, MSN dropped Inktomi. What Microsoft said at the time:

"The change to AltaVista is a better business deal for MSN. The change is no reflection on our satisfaction with Inktomi. We had an opportunity to form a deeper strategic relationship with AltaVista and Compaq. That is the primary motivation for the change," said Marty Taucher, MSN’s director of network communications.

Despite plans for a deeper relationship, in December 1999, Inktomi returned as the primary crawler-based partner at MSN Search. However, it didn’t return as the most favored provider. That went to LookSmart.

In February 1999, a five-year deal to license the LookSmart human-compiled directory of web sites was announced. By September, that information was live and coming up for most search results, with Inktomi only kicking in more unusual or obscure queries.

Beyond LookSmart, Microsoft was also doing a huge amount of work having human editors review top listings and provide query refinement options. When deals to show other search engines on the main MSN.com home page expired, Microsoft also committed fully to the new service. From my review at the time:

So there you have it — a completely revamped service. It’s one that Microsoft has so much faith in that as of Oct. 1, it stopped promoting other search services on the main MSN.com home page. Instead, the search box now queries MSN Search and only MSN Search.

"Where putting our money where our mouth is and betting the farm on MSN Search," said Bliss.

The "Bliss" being quoted was Bill Bliss, general manager of MSN Search at the time. I believe Bill took charge of the search product at the beginning of 1999, though it might have been before that. The beginning of 1999 is when I remember talking to him regularly, as he oversaw the growth of human editors that I always thought did a great job giving MSN relevant results for top queries. The query-to-listing management tools Microsoft built to help those editors were amazing.

2000-2002: More Features, More Maturity & Paid Listings Arrive

MSN Search stayed largely stable in 2000 and 2001, mainly adding new features including things like direct answers that later dropped from the service only to be rediscovered as "new." Aside from the editorial listings, Microsoft got its own paid listings program. That’s right — Microsoft had paid listings of its own before the current adCenter system. "bCentral Keywords" from Microsoft offered self-serve CPM-based paid listings on the site. But by the end of 2001, paid listings from Overture (formerly GoTo, later bought by Yahoo and now Yahoo Search Marketing) took over.

Second Era: Crawler-Based Search (2003-2006)

For years, both Yahoo and Microsoft relied on human-based listings for their main search results. Upstart Google came along with a crawler-based service that leveraged how people linked to web sites as a way to produce greatly improved results. This gave Google a single system capable of finding both the best stuff from across the web plus matches for those "needle in a haystack" searches. People noticed and started flocking to Google. Yahoo and Microsoft noticed and started thinking about shifting to use crawlers themselves. Christopher Payne led the crawler efforts at Microsoft. How that unfolded, over the years:

2003: The "Build Our Own" Decision & Christopher Payne

In late December 2002, Yahoo came out with the surprise news that it planned to buy Inktomi — the search engine that powered some of MSN Search. The search engine shopping kept going, with Yahoo announcing plans in February 2003 to purchase both AltaVista and AllTheWeb. Microsoft had looked at Inktomi and decided not to buy. It also passed on the other two players, leaving it without any strong candidates from which to acquire crawling technology, if that’s what it wanted to do.

At the time this happened, Bliss was no longer overseeing MSN Search. Instead, John Krass had taken over at some point toward the end of 2002, if I recall correctly. Bliss had moved higher at Microsoft, then went to Expedia and earlier this year jumped to start-up Klir as chief technology officer.

As for crawling technology, Microsoft did want it — but it embarked on the mammoth task of building the technology itself. Heading up the project was Christopher Payne.

Microsoft learns to crawl from The Seattle Times and Search And Destroy from Fortune, both stories out in May 2005, provide very good histories of the shift to the "build our own" strategy and how Payne was charged with it. Krass’s tenure over MSN Search was so short between Bliss and Payne that Payne really became the second person to head the Microsoft search challenge.

Microsoft started testing its own MSNBot, started massively started recruiting for employees and by mid-2003 was talking openly about building its own crawler to take on Google and Yahoo.

As back in 1997, many again assumed that Microsoft would build and conquer, in the way it built Internet Explorer and took over Netscape’s browser share. I’ve consistently warned against this view, and here was one example of that and explanation why from 2004:

Some observers see Microsoft’s coming battle as only involving Google and thus assume there will be a repeat of that other classic battle, Microsoft versus Netscape.

In reality, the battle will be against both Yahoo and Google, which both have a sizable share of the search market and their own technology. To some degree, Microsoft also is taking on AOL, another giant in the search space, though it doesn’t operate its own technology. Finally, unlike with Netscape, Microsoft isn’t getting a headstart by licensing technology to build on.

2004: Waiting, Waiting

In 2004, there’s was lots of waiting but no real delivery of the new Microsoft search engine to the mass public. One big change was that Microsoft ended its relationship with LookSmart, making MSN Search primarily dependent on the crawler results powered by Yahoo, which had acquired the Inktomi technology that Microsoft used.

In July, there were some design changes made to the MSN Search site, but that continued to provide results primarily from Yahoo. A "preview" of Microsoft’s own crawler-based results was also released that finally turned into an official beta out in November of that year.

2005: Microsoft Switches To Own Technology

In February 2005, Microsoft flipped the switch, cutting off the Yahoo results and relying on its own technology. A massive advertising campaign was also kicked off. The results weren’t impressive. The service wasn’t a Google killer or a Yahoo killer. Indeed, many felt it was still well behind being ready for prime time.

2006: Forget MSN Search, Say Hello To Live Search & Steve Berkowitz

Toward the end of 2005, Microsoft underwent a major strategy change to offer online products under a "Live" brand, one that’s been widely ridiculed. As a result, all the marketing and brand building that had been done to promote MSN Search was wasted. A Windows Live Search beta came out in March 2006, with the name later shorted to Live Search and the product itself completely replacing MSN Search in September 2006. It was just one more step in MSN as a brand seeming to be abandoned by Microsoft.

On the paid front, Microsoft’s own ad system adCenter went live in the US in May 2006. It had targeting features more advanced in some areas than Google or Yahoo. But even Yahoo’s old creaky ad system – replaced last month with the new Yahoo Panama system – beat Microsoft on the most important feature — traffic. Yahoo has more search traffic than Microsoft; Google even more. Search traffic means money. Losing it means losing money. And Microsoft was losing it. Throughout 2006, the company saw a continued decline in its search share. All the work to build a new crawler, all the marketing and effort. None of that seemed to be paying off.

Third Era: Battle For Market Share

I know it sounds weird to say the third era is about the battle for market share. It’s not like Microsoft hasn’t been battling for that for years. But there were excuses before, especially during the second era when the new technology to compete with Google was being developed. Looking At Microsoft’s Continued Long Game In Search from me in January looks back at the constant refrain from Microsoft about search being in the "early days" (as they said in 1997) and to give them more time.

Well, Microsoft has had time. Now it has the tools. It’s well out of the honeymoon period to deliver something solid. That means this era is about market share, because there are no longer any excuses to justify further losses.

It remains to be seen who will head up this third era. The stories about Payne say he’s leaving to launch a start-up company rather than as part of the reorganization that’s underway. If so, replacing him will still be part of that reorganization. And overseeing it is former Ask.com chief Steve Berkowitz, who was hired by Microsoft to oversee the MSN Online Business Group.

Marketing and business development of Windows Live products, such as search, fall within Berkowitz’s group. He was fairly quiet after going into Microsoft. Several months after being there, he was talking much more, as in a New York Times profile where he described Microsoft as a "cruise ship" that can be slow to turn:

“I’m used to being in companies where I am in a rowboat and I stick an oar in the water to change direction,” said Mr. Berkowitz, who ran the Ask Jeeves search engine until Microsoft hired him away in April to run its online services unit. “Now I’m in a cruise ship and I have to call down, ‘Hello, engine room!’ ” he adds with an echo in his voice. “Sometimes the connections to the engine room aren’t there.”

Now Berkowitz seems to be not just steering the ship but changing the crew. Microsoft Shifts Windows Vista Executive to Search from Bloomberg covers how Brad Goldberg, an executive in the Windows side of the Microsoft house, is being shifted to oversee marketing for Windows Live Search. Mary Jo Foley at All About Microsoft writes further on reorganization rumors and changes. Techmeme has a good roundup of others writing on the reorg issue, as well as a round-up here on Payne’s departure.

Part of that reorg round-up is a fresh call from Between The Lines writer Larry Dignan for Microsoft to buy Yahoo. Well, the two have talked before, though about Microsoft owning a part of Yahoo, not the entire thing. As I wrote about that when news of this came out last year, there are pros and cons:

Bringing [Steve Berkowitz] on was a big sign that what Microsoft has been trying to do internally hasn’t been working — and so something radical such as an Ask or Yahoo acquisition might be in order.

The big downside is that such an acquisition would give Microsoft yet another brand to confuse consumers with. After spending hundreds of millions of dollars over the years to push MSN, they’ve now shifted things behind making the stupid Windows Live brand their flagship….

So Microsoft’s already coping with the confusion of two major brands. Adding in Yahoo further confuses matters, unless they perhaps make a brave, bold move and put everything behind the brand leader in the space, Yahoo.

An acquisition made tons of sense before Microsoft built its own crawler and ad system. Now it has both. Getting Yahoo last year might have saved Yahoo from upgrading its own ad system, since it could have used the new one Microsoft built. But Yahoo has finished that work, as well. So a merger or acquisition makes less sense — and the brand confusion potential remains significant. But stranger things have worked.

As a reminder, last year CEO Terry Semel was pretty bullish on Yahoo’s future against Microsoft:

“My impartial advice to Microsoft is that you have no chance,” Mr Semel said. “The search business has been formed.”

Some good news for Microsoft. Compete reported yesterday of a significant uptick in Live Search’s traffic. Compete hasn’t been in the ratings space as long as competitors comScore, NetRatings and Hitwise, however. So the uptick is noted, but it will be more reassuring if we see it also with one or more other services — and especially continuing over the coming months.

Related Topics: Channel: Strategy | Microsoft: Bing | Microsoft: Business Issues | Microsoft: Employees | Microsoft: General | Stats: History

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • MrBill

    Hi there Danny, my ears were burning so I thought I’d clarify a few things. Thanks for the kind words.

    You got the chronology of my tenure at MSN Search more or less right. I actually started in the summer of 1998 but we launched a pure Inktomi version in September 1998, about the time we licensed Looksmart’s directory. In early 1999 we launched the server software with our own relevancy ranking against the Looksmart directory and appended Inktomi results when there weren’t enough relevant results to fill the results page.

    A few more tidbits that you and your readers may find interesting…

    * The decision to switch to AltaVista was an awful one, which I objected to strenuously, but I was overruled. The good news is that throughout it all, Inktomi was a great partner and we never turned them off, so when we finally were able to switch back, it was very quick and easy.
    * I think it was about 1999 or 2000 when we first started noticing this search engine called Google growing in MediaMetrix reach about 1.5% PER MONTH. That’s almost unheard of. I knew they were on to something, and alerted MSN management (which changed every few months) every six months or so. I was always told “Search is not core to our business, Google is not a competitor, Yahoo is not the competition, AOL is the competitor to beat, subscription services is how we’re going to win.” Hailstorm anyone? Please.
    * It was even worse for John Krass, who was publicly castigated in large meetings at least twice for suggesting that MSN should compete against Google. We had plans to release an MSN Search toolbar twice and both times it was cancelled by senior MSN management. John did a yeoman’s job of trying to convince his management to pay attention to Google, but they continued to ignore Google.
    * In 2002 they re-organized MSN Search yet again and my new boss was Kai-Fu Lee — yes, the same one who’s now at Google as President of Google Chinea after being sued by Microsoft for defecting (after I left for Expedia). One of the reasons I left MSN Search was that Kai-Fu told me, it’s pretty clear Microsoft is not going to invest in web search, so you might as well work on search technology for a product that Microsoft does care about (Windows). He was right, so I did.
    * If there’s one thing I regret, it’s that I didn’t bypass MSN management and communicate the importance of web search to Bill Gates and Steve Ballmer. I was a good soldier and didn’t go above my boss’s head, which in hindsight was a mistake. That said, I would have done it differently — Microsoft could have been the one to buy Inktomi and Overture instead of Yahoo, and the world would be much different now for both companies…

    Microsoft has a long history of eventually getting it right, even if it takes a while. Hopefully for Microsoft’s sake, they will.

    Bill Bliss

  • http://searchengineland.com Danny Sullivan

    Thanks, Bill — was going to track down your email address and see if you’d go public with some of these things. You saved me the time!

  • http://www.thesage-speaks.com TheSage

    I think MSN Live has a LONG WAY to go compete in the search space. There index seems to be just a fraction of Yahoo or Google’s. I am hopeful they can crate something good. By all means, they have plenty of cash backing them.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide