Try as they might Yahoo is having a tough time convincing people that it’s still “in search.” On Yahoo’s earnings call earlier this week CFO Tim Morse, who led the call because CEO Carol Bartz was ill, used her analogy to describe Yahoo’s new way of positioning itself around search:

The next revolution isn’t with the algorithms that provide results, it’s in creating a better, more personally relevant search experience. This is where we’ll differentiate ourselves and compete vigorously without the billions required to keep up in the arms race that generating search results has become.

Let me give you an analogy that Carol has been using to explain this point. Consider basic search to be an Intel chip. An Intel chip is used in Dells, HPs and Macs to provide the computation needed to operate them but the differentiation between these products isn’t at the chip level, it’s in the different user experiences that are provided on top of them. It’s the same for us in search. We’ll innovate on top of the results that are provided to us by Microsoft.

While a clever way to discuss Yahoo’s new role, vis-a-vis Bing (the algo “chip”) it doesn’t seem to be convincing many industry insiders.

Yesterday at the Web2.0 event in San Francisco Google co-founder Sergey Brin made a “surprise” visit and talked about a range of things. Dow Jones newswires captured some of the discussion. Specifically Brin lamented the “abdication” of search by Yahoo. Here’s TechCrunch’s version of the relevant part of the exchange between Brin and interviewer John Battelle:

JB: Do you like Bing? You a Bing users?

SB: I use all search engines out there. Bing reminds us that search is a competitive market. There’s Powerset that Microsoft bought. There’s Cuil. There’s a lot of interesting stuff going on. It’s a shame Yahoo is abdicating.

JB: They would say they’re not.

SB: Sorry that was my impression.

JB: Do you have a comment on Microsoft/Yahoo search deal?

SB: I shouldn’t comment on that. But Yahoo had some interesting things, they should stick with it.

The Cuil folks should be pleased that Brin gave them a shout out.

Yahoo has now gone live with its new search format. There’s nothing radical or “game changing.” However, there are some nice upgrades and improvements. Most prominently, it features a new left column that allows users to filter results by Search Monkey content providers or refine by related concepts. It also features more prominent placement for Search Pad and an expansion of Search Assist. Yahoo says it has also improved image and video search and says speed and performance are better across the board.

Yahoo previewed these new features previously and Danny discussed them at some length in an earlier post. These changes are now in effect in the US, UK, France, Spain, Mexico, and India and will roll out more broadly to other markets in the near future. Overall there’s more visual appeal to the new format and the Search Monkey refinements are useful in most cases.

Below is an example search for London. Shortcuts remain at the top of results, now the center column:

If we click the “Yahoo Travel” filter in the left column, we get the following results without entering a new query:

Further down in the left column you can also refine by “weather” (not a Search Monkey filter, but a related “Search Assist” concept):

Search Assist continues to be a prominent feature of the overall search experience. And Search Pad gets more visibility:

The newly launched search format generally mirrors the new Yahoo homepage with its brand-centric left column:

With this new look and feel Yahoo Search moves somewhat closer to Bing, ironically. Bing has seen its market share slowly creep up and is probably a greater threat to its partner Yahoo than it is to Google.

In the relatively near future the actual results on Bing and Yahoo will be the same (assuming regulatory approval of the deal). Thus Yahoo will continue to need to make changes and upgrades to the interface and overall user experience to prevent a loss of usage and market share.

I haven’t systematically tested Yahoo’s new search against Google and Bing. However I generally like the new appearance as well as the Search Monkey features.

One thing that Yahoo should consider is a personalized left nav/column (a la the new home page) where users can select favorite Search Monkey filters (e.g., Yelp for local, Kayak for travel). Indeed, the more prominent appearance of Search Monkey in the new Yahoo Search may encourage more sites and publishers to participate in the program.

We hope that these changes are part of an ongoing series of upgrades to Yahoo Search, although many continue to believe that with the outsourcing of its back end and index to Microsoft, and exodus of key search engineers, that Yahoo has effectively given up on search. New Yahoo CMO Elisa Steele tried briefly to rebut that idea in her IAB MIXX keynote speech announcing the new Yahoo branding campaign.

Here’s more on the new search changes from the Yahoo Search Blog.

Is it possible that, say a couple years from now, you might visit Yahoo.com and its Bing-powered search engine, but actually search Google, instead?

It seems so. PaidContent reports on Yahoo’s appearance Wednesday at the 2009 Deutsche Bank Technology Conference, where Burke Colligan, a senior director for Yahoo Front Doors — the company’s user experience group — said it’s possible the company may someday allow Yahoo users the choice to search a different search engine from the Yahoo home page.

“Right now, we’re not doing that but long-term that may not be out of the question. We want to provide the best experience for the user and, in some way, if that was the best experience for the user we might be open to that.”

When Yahoo and Microsoft announced their search deal in late July, the primary story was that Microsoft would provide Yahoo’s search results, but Yahoo would continue to “own the user experience.” One suspects that Microsoft didn’t think about the possibility of that user experience including a Google search option.

During the Yahoo-Microsoft conference call with CEOs Bartz and Ballmer and subsequent discussion with Microsoft’s Yusuf Mehdi and Yahoo’s Hilary Schneider we heard repeatedly that although the two engines would share a single index and Microsoft would incorporate elements of Yahoo Search (e.g., Search Monkey) into Bing, Yahoo would continue to be different and vital in search.

Freed from the cost ($425 million reportedly) and ongoing demands of the back end, it would “innovate” around the interface and search user experience.

Many people in the industry were skeptical and  shrugged it off as something akin to wishful thinking or putting a brave face on Yahoo’s exit from the search business. (Danny wrote “A Search Eulogy for Yahoo” thereafter.) But Prabhakar Raghavan, in charge of Yahoo Labs, discussed on Friday, how the company might incorporate Twitter and real-time search into Yahoo results. He again made the case that Yahoo would continue to innovate and present a compelling search user experience.

“In terms of satisfying user intent, the hard work and in some sense the bigger growth opportunities for differentiation are not the back-end of crawling and indexing, but really surfacing and assembling content the right way to satisfy user intent,” he said.

Real Time search is an increasingly popular online activity where Yahoo’s approach to search could provide a compelling user experience, Raghavan said.

It’s theoretically possible but one of the practical challenges will be recruiting and retaining top search talent, as the NY Times points out:

Yahoo will lose some of its most talented engineers to Microsoft and as many as 400 employees through layoffs. The deal also undercuts years of investment around search technology. By selling the technological crown jewels, the company may lose some of its high-tech credibility among employees and others in Silicon Valley, as well as among customers.

Investors, who had been expected to cheer the deal, punished Yahoo for the terms of the transaction. On Thursday, the day the deal was announced, Yahoo’s stock declined roughly 12 percent — although it has largely recovered since then. Financial analysts and investors saw the deal terms as something of a fire sale. According to the Times article:

Microsoft offered $46 billion to buy all of Yahoo. Analysts estimate that the new deal — involving what many people saw as Yahoo’s most important asset — is worth only around $4 billion to $5 billion.

The deal was largely motivated by Ballmer’s persistence and Yahoo’s inability to compete with Google and Microsoft in search at the required levels of investment. There was also lots of additional pressure coming from the market. All of these things, among a couple of others, perhaps made the deal inevitable.

The deal isn’t done until regulators in the US and EU approve it. That process is not, as they say, a “slam dunk.” But let’s assume that it does go through. Yahoo can’t simply ignore search. It will still be forced by the market to pay attention to search volumes and monetization (RPS).

To that end, Yahoo will in fact need to invest in the user experience to maintain its position. If it doesn’t it will lose share to Google or Bing. If that starts happening Bartz and Yahoo will be under intense pressure. So the notion of “innovating around the user experience” isn’t simply an aspiration for Yahoo, it’s a necessity.

A sign of things to come, perhaps? Based on this screenshot posted today on Flickr, Yahoo appears to be testing a new layout for its search results pages.

It looks very much like the new Yahoo homepage that launched a couple weeks ago, which suggests this may be soon become the default layout for Yahoo’s search results pages. The Yahoo logo, which appears in red and on the right side of the current search results page, is in purple and moved to the left here. The SearchPad feature is also moved from right to left. This new layout also mimics Bing slightly, with the list of “Related Concepts” appearing in the left-side column.

This seems to be a limited test; none of the SEL editors who tried today were able to see this new search results page layout.

When I heard about the call with Yahoo yesterday, which was to feature “a significant announcement,” I thought: “Bing, here it is.” When it turned out the call was about a formal announcement of the new Yahoo homepage I was disappointed; it seemed anti-climactic.

The existence of a forthcoming Yahoo homepage redesign has been known for months and screenshots of earlier versions have been seen previously in several places online. However, the new Yahoo homepage, which launches around 4:30 Eastern today, turns out to be both interesting and practical. The embargo was broken yesterday so immediately everyone rushed out their stories and there’s already been considerable coverage.

Here’s the Yahoo homepage as it exists this morning:

Below is the new homepage. It will not immediately replace the one above but be an opt-in choice for those interested in the near term:

Overall the page is less cluttered, there are fewer buttons and modules and search is somewhat more prominent. In the current Yahoo homepage search seems to exist outside the main visual field, whereas in the new design it’s more “visible” and integrated into the rest of the page.

The left column goes from being a static list of Yahoo properties to a customizable menu that effectively turns the homepage into an RSS reader or “dashboard.” Borrowing from MyYahoo (which will continue on as it is), Yahoo allows users to add or remove widgets or applications. Those include third party sites.

Yahoo has created a range of these widgets for launch, but users can themselves add any site with RSS feed capability. And third party developers will soon be able to add their own applications to the gallery. In this way it conceptually brings together the homepage and what Yahoo is doing with SearchMonkey.

Typically users don’t do much customization. I raised this question with Yahoo SVP Tapan Bhat who was presenting the new homepage. He agreed and said that Yahoo will recommend widgets to users based on their browsing and click-stream behavior. The process of adding or removing “applications” is simple and basically involves a single click.

As users mouse over the individual “My Favorites” applications a window opens that allows them to see the site or content. Here’s the official Yahoo screenshot with Facebook as an example.

Users can update Facebook (or Twitter) from this window. On the right side of the window is a new, contextually relevant ad unit that can be targeted to the content of the page. One example Yahoo gave on the call was for a movie opening targeting users looking up movie showtimes from a Yahoo movies application.

I haven’t been able to use the new homepage but it appears Yahoo has done a nice job balancing “push” and “pull,” adding simple but useful customization without diminishing the broad reach that the page offers.

Users will also later be able to adjust the news that they see on their homepage on a sliding fun-to-serious scale:

In terms of search, Yahoo will start “bucket testing” (random testing) a new look and feel for search results that conceptually mirrors what it’s doing on the home page. Below are screens I captured from the presentation. In the left column are (Search Monkey) widgets that effectively become search filters. The second example below shows the same search only showing results from YouTube:

Similar to the homepage favorites, the benefits of this approach rely on people customizing search and adding filters. However it’s pretty interesting on multiple levels. This search page will not be widely available today when the new homepage launches because it’s being selectively tested. In fact it’s possible it may never roll out.

How all this may or may not play with Microsoft if or when a search and display ads deal happens is uncertain. The question was asked on the call about how Bing might tie into this page. Bhat, as one would expect, demurred and declined to say anything.

Finally, the new homepage will make its way to mobile fairly soon. The recently launched mobile portal was ahead of the PC site in terms of customization. But the changes should make it simpler and more useful still.

The new homepage will roll out first in the US, followed by the UK, India, France and other countries thereafter.

If Microsoft brings “boatloads of money and the right technology,” Yahoo would be willing to strike a long-rumored search deal. So said Yahoo CEO Carol Bartz at the D7 Conference today, where All Things Digital’s Kara Swisher interviewed her on stage. Bartz followed up that statement by saying Microsoft would need “biiiig boatloads of money” if it wanted to buy Yahoo entirely.

That’s just one of the headlines coming out of Bartz’s appearance, but certainly not the only one. According to coverage from Barron’s, the Wall Street Journal, TechCrunch, and All Things D here are other highlights:

• Yahoo is interested in video advertising and social networking, and would consider acquisitions in those two areas.
• Bartz said Google is “very good” in search and with its maps/Earth service, but Google doesn’t have the same reach & positioning that Yahoo has. “We are a place people come to be informed. Google is a place people go to do search.”
• Bartz listed Yahoo’s three most important properties/focuses as its home page, search, and its mail/news/sports/finance sites.
• She emphasized Yahoo’s deep content several times, at one point saying Facebook doesn’t give users news, stock quotes, and “a lot of things.”
• Yahoo needs to make its user experience simpler and more customizable.
• Bartz says Yahoo has 76% reach in the 18-24 year old demographic, and half the mobile users in the U.S.
• Yahoo’s new home page should debut this fall.
• Bartz wants Yahoo to get more local. On its new home page, users will be able to access highly local apps and information. She mentioned wanting to show high school football “Flip videos.”
• The need for better organizational structure at Yahoo is one of the main things that led her to accept the CEO position. “Yahoo is on its way to being a lot simpler and therefore faster on its feet,” Bartz said.
• Bartz doesn’t expect any more layoffs.

There’s more coverage of the Bartz interview on Techmeme.

Yahoo continues to develop a new home page, and today has shared a sneak-peek at the latest design that’s being tested internally.

The “My Favorites” column on the left side of the page seems to be where the big changes are happening. Yahoo currently lists more than a dozen of its own properties in this space, but the new home page will feature a dashboard of sorts that users can customize with many of their favorites sites — including non-Yahoo sites like eBay or the Wall Street Journal. Users can also keep track of their friends social activities via a new Yahoo Updates app. And if you have multiple webmail accounts — say, Yahoo Mail, AOL Mail, and Gmail — you’ll be able to access them individually from the home page. It’s not unlike Yahoo’s new mobile home page, which offers similar types of content customization.

Yahoo says the new home page design is still a work-in-progress and is being updated frequently based on user feedback. No word on when a final version will be ready, but today’s announcement suggests there will be more updates “in the months ahead.”

There are arguably two parallel tracks in the “what will succeed Google?” meme. An emerging one is “will the feds intervene to block Google?” The second is much older and involves the hypothetical “semantic web” or sometimes “deep web.” Essentially this asks, “who will do search better than Google?” (A third line of argument might surround “social search.”) Two pieces in two days in the NY Times reflect the two former strands.

The first article is Everyone Loves Google, Until It’s Too Big and picks up on the monopoly discussion. I responded over the weekend on my personal blog Screenwerk. The second and subject of this post is Exploring a ‘Deep Web’ That Google Can’t Grasp.

From the “Deep Web” article:

Beyond those trillion pages [indexed by Google] lies an even vaster Web of hidden data: financial information, shopping catalogs, flight schedules, medical research and all kinds of other material stored in databases that remain largely invisible to search engines.

The challenges that the major search engines face in penetrating this so-called Deep Web go a long way toward explaining why they still can’t provide satisfying answers to questions like “What’s the best fare from New York to London next Thursday?” The answers are readily available — if only the search engines knew how to find them.

Yes, an engine that can mine all that data and present “answers” to users would be quite exciting. However, as the article points out, Google is also investing in trying to mine more of that so-called “deep web” itself.

There have been many extravagant claims by and about semantic search engines (e.g., Powerset, bought by Microsoft) and deep web projects in the past (Chris Sherman has much more perspective on this than I). But so far, none of them have really borne fruit.

A quote from the “Everyone Loves Google” article is also relevant here and I believe correct about what changes might make a difference at least in the near term:

“Whether we’re slightly ahead or slightly behind Google in core relevance is not a game changer in search,” said Prabhakar Raghavan, Yahoo’s chief search strategist.

Yahoo’s best opportunity, Mr. Raghavan said, is to offer radically new ways of presenting information that will help users finish whatever it is they started before the search, like finding a job or buying a plane ticket. “People don’t want to search; it’s a digression,” he said. “They want to complete a task.”

Search results pages right now are terribly cluttered (and flawed in my opinion). But doing something that truly delivers on the “complete a task” metaphor is challenging in myriad ways:

• Technology/algorithm
• Interface design
• Political: picking winners and losers from among similar sites to a much greater degree than today

Mobile is also an interesting lab for PC-based search. The limitations of the mobile “form factor” and the greater need/immediacy regarding the information will likely hold some eventual lessons for online search.

Microsoft and Yahoo (assuming it doesn’t sell search to Redmond) will continue to make improvements in their algorithms, indexes and interfaces. The more competition the better because search will only become more important as the “deep web” is unlocked.

The Yahoo corporate blog announced that phase one of the Yahoo home page test is about done now. Yahoo has taken in a lot of feedback since launching a limited test of the new home page back in September 2008 and has decided to try some new things.

Yahoo has decided to add more third-party apps from eBay, Forbes.com, Wired.com and others. They have decided to “beef” up the Sports and Finance apps, by providing schedules, team standings, blogs links and more, plus one-click access to your stock portfolios and stock quotes. Plus, now you can customize the apps by adding and removing them, as you see fit.

The most significant change is that Yahoo has lightened up the page, based on user feedback. Here is a picture of the new look:

The old Yahoo’s “new” home page looked like this: