Microsoft Live Search Core Relevance Program Management Director Eytan Seidman Moves On

Eytan Seidman, Program Management Director over core relevance for Microsoft Live Search, is leaving Microsoft (and Seattle) after six and a half years and heading to New York to work on a start up with his brother. I talked to him about life at Microsoft, the birth of Live Search, and singing karaoke in New Orleans.

And don’t miss the video tribute to Eytan at the end of the article!

The core relevance team that Eytan managed includes crawling, indexing, and ranking for Live Search. And while he acknowledges that Live Search still has work to do in relevance, he’s happy with the amount of progress they’ve made in such a short period of time. He points out that Microsoft only launched its own search index in 2003 and already has results that — in his estimation — beat Yahoo! and Ask.

Early Days
Eytan started at Microsoft as an intern for bCentral in the summer of 2000. He built customer applications for small and medium-sized businesses and came back full time to work with the team in the summer of 2001 after graduating from UPenn with dual degrees in History and Computer Science. He says the History degree has helped him with all the writing he’s done, as well as with looking at things over time in context, rather than as a single point. (His favorite part of history? World World II.)

He joined around the time Microsoft acquired Great Plains, so he spent time in Fargo, North Dakota and worked on integration of back office financial applications. He worked on the team for two years, shipped the first version of the Microsoft Dynamics CRM product in February 2003, and transitioned from the team in June of that year. He says he’d recommend what he did next to everyone.

Everyone Should Sing Karaoke
And what was that? He traveled around the United States, hitting such highlights as the Hoover Dam (which contains 3,250,000 cubit yards of concrete), the Grand Canyon, a Subway at the Loiusiana/Texas border, and a karaoke bar called Cat’s Meow in New Orleans (he sang Ace of Base).

The Beginning of Live Search
After all the driving and singing, he returned to Microsoft to work on search. At the time, Microsoft search was located at msn.com and was powered by Inktomi. Yahoo! had purchased Inktomi in December of 2002, and faced with the fact that their supplier had been bought by a major competitor, Microsoft began work on its own search engine in January 2003. Eytan was the first Program Manager on the new search project and the team started out with just five developers. When he joined the team, they had just passed their first major milestone of crawling 10-20 thousand documents. “We didn’t yet know how to serve or rank, but we had started crawling, so we were excited,” he recalls.

[Note at this point in the interview, we break to find Ace of Base songs on iTunes so Eytan can relive his karaoke glory days.]

The small team kept working on the new search engine (code named Monarch — maybe that’s where the MSN butterfly comes from!), while the existing team kept up MSN Search. This new team had help from experts in Microsoft Research (including Marc Najork, Mark Manasse, and Mike Burrows), and the team learned a lot about how search works (and just how large of an undertaking it is).

[He’s now switched iTunes to Nelly Furtado.]

Making It All Work
The new search team put together an end-to-end system of crawling/indexing/ranking/and serving with a 100 million document index they called “Little Dog.” Even though the index was small, it all worked and it was the first time they’d had an end-to-end search engine that was entirely their own. By early 2004, Little Dog was up and running. Eytan said these were some of the best times they had in search. A small team of developers, testers, and program managers had managed to build a search engine, even though they had bumps along the way. He recalls, for instance, learning about crawling infinite calendars and using up server bandwidth. He said people would sometimes send them bandwidth bills. As part of his work helping with the design and architecture of the crawling system, he monitored the [email protected] email address, which as left in server logs when MSNbot would crawl.

An Early Search Preview
They spent the next year getting ready to launch publicly. They hoped to launch in fall of 2004, but things took longer than expected. They did launch MSN Search Technology Preview in October of 2004, with a subset of the index. He said the launch date was a crazy all-nighter of network problems, getting slashdotted, then getting the server back up. He said relevance was terrible but at least it was a start. (The technology preview was actually available earlier in the year. In July, WebmasterWorld members spotted it. msndude said at the time, “A little while back I mentioned that we were using the data gathered from the recent crawls as part of a preview of an algorithmic search engine that we are building” and said that they had about a billion documents indexed at that point.)

MSN Search Launches
The new search engine launched publicly as MSN Search in February 2005 (and was rebranded Live Search in late 2006). The team at that point had grown to around 100 people and had support all the way up to Bill Gates.

[We stop at this point to look for some Nirvana, as Eytan mentions that it’s the 20th anniversary of the band forming. This nostalgia for grunge is proof that he’ll miss Seattle.]

After the initial launch, Eytan spent some time as the program manager for the crawling system, working with large sites such as amazon.com and ebay.com. He also did some release management, since at this point, he knew the product inside and out. He recalls a product review meeting with Bill Gates where he shows a ramp up from 0% to 100% between December 2004 and February 2005. Their plan to create a search engine entirely their own had worked and had only taken 18 months. They found that their relevance was better than Inktomi, so even though they had lots of work to do, they were excited at their progress.

Eytan kept working on more parts of the product and in the summer of 2005 started leading a program management team in core infrastructure. He says they made lots of incremental progress between summer 2005 and late 2006. Here he calls on his background in history. “To put things in context historically, Google had been around for more than six years when we launched, so we were already that far behind.” So, while he knows that they didn’t launch initially with the largest and most relevant search engine out there, they’ve been dedicated to building improvement and while it’s not the kind of thing you can build overnight, the progress has been amazingly fast. “Relevance is now up there with Google and Yahoo!,” he notes.

The Importance Of Webmaster Relationships
Late 2005 is also when you may have started seeing Eytan speaking at search conferences. Eytan has been a vocal advocate for webmaster needs inside of Microsoft. Recently, he said:

“Webmasters are a critical piece of our success. Fundamentally it is webmasters who provide us the content that makes our engine useful to end consumers.”

Microsoft’s efforts with the Live Search Webmaster Center are proof of this perspective.

A Focus On Relevance
Eytan has most recently been in charge of relevance (which includes crawling, indexing, and ranking) and relevance measurement. It makes sense that all of these things are part of relevance since all of these factors play into the results of a query. A search engine might have poor relevance because the best page hasn’t been crawled or indexed or because it’s not ranked appropriately. Eytan notes that of these major components, ranking is the more complex problem over time. Once you have a good method for crawling and indexing documents, it’s mostly a matter of scale. But ranking is trickier. He says that it’s part engineering and part research — your experiment and see what works.

In a recent interview with SEOmoz, he talked a bit about relevance measurement:

“we do look at multiple things including “blind” tests as well as data of how our users are engaging with our live site. In terms of satisfaction, the improvement is across the board. One of the things we highlighted at our recent Searchification event is some of the neat work that we did around “query intent.” That is, we are doing a ton of work to better understand what people really mean when they enter a query.”

Eytan’s team is looking at end-to-end scenarios to make relevance better. For instance, why does Live Search underperform on local queries? They’re tackling that age-old search problem that people query the way they think, not the way documents are written. How should stemming come into play? You might think this stuff is trivial — after all, Google’s been looking at these problems for years. But it’s not trivial to Google and it’s certainly not trivial to a search engine team that’s been around a third as long.

The Future Of Live Search
What’s the future of Live Search, particularly now that one of its key team members is moving on? Eytan says the team is well-positioned for success, that Microsoft takes a long-term view and isn’t only in it for the short-term win. He points to Xbox as an example of a project on which Microsoft was willing to take an initial hit on that now is a half billion dollar business. Windows Mobile is another example of a project on which Microsoft stayed with for years before it became profitable. He also notes that Microsoft isn’t afraid to get into a large market with a lot of competition – take for instance SQL Server and Exchange. He says SQL Server now has a 60% market share, so in retrospect, it seems obvious they should have gone after that market. But at the time, it was crazy.

Search is the same. There’s no single, magical answer, he says. You just keep working on it, have focus, and remember that you’re in the business for a reason – to add value to the space. Microsoft understands that something like search isn’t easy and fast and they’re dedicated to putting in the time that it will take. And that support comes all the way from the top. Bill Gates meets with the Live Search team several times a year.

(News of a potential Microsoft acquisition of Yahoo! came after our interview, which adds a whole new dimension to the future of Live Search.)

And what’s next for Eytan? He’s not saying much about his new startup, but he will say that it’s exciting to work on something new. He says he loved his time at Microsoft – it’s an awesome place with awesome people, but the time has come to build something of his own.

He says Microsoft prepared him well for life with a startup. He came to understand the importance of the user experience when working on Microsoft’s CRM product. (And the original concept he worked on is still in use.) He says that in many ways “the “user experience is the product” and a compelling experience for users is key. He saw this, for instance, when working with the Image Search team and watching how they saw how image search was different from web search and added functionality such as with infinite scroll and the hover experience.

He also has a better sense of the importance of search with any kind of web application. He feels search will become a critical part of everything we do online and customers will expect a great search experience. For instance, Amazon isn’t a search company, but quality search is core to what they do. He says, “What I found out on search and continue to believe is that search in every sector is complex and that folks who do a good job at in their space will be able to differentiate themselves from the competition.”

Both Microsoft and the search community will miss him.

Said Justin Osmer, Live Search Sr Product Manager (and the one who twisted Eytan’s arm to get on stage at SES for the first time):

“Eytan has always had a customer-centric view of search…what will make it better for our customers…and many of his decisions and recommendations reflected that view. He is a strong advocate for the webmaster community and for those you remember search champs have him to thank as being one of the drivers of that industry-first program. We will certainly miss his bushy hair and big smile around the halls here at Microsoft, but I am sure this isn’t the last we’ll hear of him.”

And now, a video in two parts, starring many of Eytan’s friends from search: