• The web and image search results are now sourced from Microsoft’s search index
• Yahoo branding attribution is now required (the first version allowed white labeling)
• Developers are charged based on query volume (the first version was free)
• the key terms/query suggestions feature has been discontinued

Upgrade Process

Yahoo says that it’s taken 7-10 days for developers they’ve worked with to upgrade their services from version 1 to version 2. Version 1 of the API will be going away on July 20th, so all services will need to be upgraded by then. They don’t have an upgrade guide that outlines what to look for when modifying services that use BOSS but say they’ll be available to answer questions in their Yahoo user group.

Branding Requirements

The first version of BOSS allowed white labeling, but now, services will have to clearly state that results are “Created with Yahoo Search”, use the Yahoo logo, and link back to the Yahoo BOSS site. I clarified with Yahoo that the service itself requires this branding, but if the service uses results from multiple services, the results from Yahoo BOSS don’t have to be notated separately. The branding has to be included only once and can be on the About of FAQ page of the service.

What else is new? The web search results will be coming from Microsoft’s Bing index rather than Yahoo’s. A earlier post noted, “BOSS will continue to be a RESTful API for web, image, and news search. Core web and image results will eventually be powered by the Microsoft search platform.” I clarified with Yahoo today that in the new version about to be released, web and image results will be coming from Microsoft and news results will be coming from Yahoo.

(Although Yahoo’s U.S. web search results now come from Microsoft, Yahoo will continue to crawl the web until at least 2012, when the international transition is scheduled to be complete.)

And this version includes something else not in the previous version: Yahoo branding requirements. Originally, Yahoo didn’t require BOSS implementations to include Yahoo branding, as one of their key stated objectives was to disrupt the search market and power startups. The idea was that startups could use BOSS to power their innovative search products and create their own brands, rather than build on Yahoo’s.

However their latest post says “We want you to be able to leverage the Yahoo! brand to help signify the valuable platform on which your applications are built.” This could be interpreted as a step backward as white labeling was a key value proposition they gave as an improvement over the previous Yahoo search API. As their FAQ currently states, “You can take BOSS results, blend in your own secret sauce, and build a search engine of your own design, all without required brand attribution.”

How well has BOSS done with their mission of disrupting and innovating the search space? Yahoo would only tell me that “there are hundreds of developers using BOSS today”. The Yahoo BOSS web page lists several examples, including hakia (which has their own index and ranking system, so uses BOSS to augment their results) and OneRiot, which has shut down its search engine and is now using the Topsy API. (Duck Duck Go uses Yahoo BOSS today, along with other data sources.)

Yahoo BOSS messaging as of late has focused on making “business and economic sense“. But it’s unclear if the pay structure will be well-received by developers when combined with the branding requirements.

It’s also unclear what the advantage of this new version of BOSS will have over using the Bing API directly. The Bing API also offers unlimited use and flexibility of ordering and blending results. The Bing API terms of service do require that ” if you interleave data from any source other than the API with data from the API, clearly differentiate the respective sources.” With the new branding requirements from BOSS, the display requirements may be similar.

News is traveling fast about a leaked slide from a Yahoo “all hands” meeting earlier this week that listed several products — including Delicous and AltaVista — to be “sunsetted.” Yahoo confirms that Yahoo Buzz will be closed, as will the Traffic APIs. Delicious seems likely to go, too.

Here’s the statement we received from Yahoo:

Part of our organizational streamlining involves cutting our investment in underperforming or off-strategy products to put better focus on our core strengths and fund new innovation in the next year and beyond. We continuously evaluate and prioritize our portfolio of products and services, and do plan to shut down some products in the coming months such as Yahoo! Buzz, our Traffic APIs, and others. We will communicate specific plans when appropriate.

I asked again if this included Delicious and was told:

We’re not commenting on Delicious specifically at this point.

The Sunset Slide

The news appears to have been broken first by Eric Marcoullier, who tweeted out a picture from an internal slideshow that was given to Yahoo employees:

You can see a larger version of this here. Marcoullier was previously a Yahoo employee, having come over when Yahoo acquired MyBlogLog — a service he cofounded. Several Yahoo employees seemed to confirmed the image via Twitter, as AllThingsD reports, as did Yahoo Chief Product Officer Blake Irving, when he tweeted about firing the source of the picture:

Can’t wait to find out how you got the web cast. Whoever it is, gone!

Goodbye To Yahoo Buzz

As you can see, Yahoo Buzz is on the list of “sunset” products, those that will be closed. When Yahoo Buzz launched in Feb. 2008, it took over the name from a previous trends-based product and was positioned as a type of new Digg with close links to publishers.

So Long, AltaVista & AllTheWeb

Also on the list are AltaVista and AllTheWeb. Both are search engines that Yahoo inherited from Overture, which in turn bought the services in 2003. AltaVista used to be a giant of the search world but has long since been eclipsed by Google. Neither gets much traffic, though both have links from across the web. Indeed, all those links still keep AltaVista ranking well on Google for searches on both “search engine” and “search engines.”

Closing those services makes sense to me, in that they don’t offer anything that Yahoo doesn’t already offer. These days, they’re both powered by Bing — as is Yahoo. The only reason to maintain them is for the extra “shelf space” they get from across the web. If they’re closed, and redirected to Yahoo, the traffic will be maintained in the short term. In the long term, some of that legacy traffic will finally disappear.

See You, MyBlogLog

MyBlogLog enjoyed a brief rise of fame as a social widget that could be installed on your blog. Yahoo bought it in 2007, and Yahoo Acquires MyBlogLog & More On How It Works is our article on how it worked at the time. I found it soon became pretty spammy and stopped using it, myself. These days, I think it’s fair to say that Facebook Connect and the “friend” widget it allows has left MyBlogLog behind. I won’t miss it, for one.

Coincidentally, Marcoullier has a sequel to MyBlogLog that just came out in beta testing today: OneTrueFan.

What’s Next For Delicious?

As for Delicious, I’d say it’s one of the survivors of the social bookmarking world. It continues to have a strong, loyal following. I can see Yahoo making a hard decision that it’s not feasible to maintain, for some reason. But it’s a terrible mistake to have let the news leak out this way. I think it’s just going to create doubt among those loyal users about the future of their bookmarks, what do they do next and whether it’ll be closed permanently, closed to new users or — hopefully — shepherded to a new company that will maintain it.

Postscript by Matt McGee: In that last sentence, Danny is wondering what many Delicious users are also wondering: Would or could Yahoo sell Delicious? Could it give Delicious over to the community as open source code? There was a quick conversation about that on Twitter a little while ago between Hunch co-founder Chris Dixon and Joshua Schachter, the creator of Delicious (who sold it to Yahoo in 2005). Schachter told Dixon that both scenarios are probably unlikely:

Could the loyal Delicious community step in somehow? Just a couple months ago, the community came together when news spread about the demise of Xmarks — a popular web browser bookmark syncing tool — with many volunteering to begin paying for what had been a free tool. Three days later, Xmarks announced that interested buyers had come forward and Xmarks would be saved.

There’s an obvious and huge difference between Xmarks and Delicious: One was owned by a spunky, little startup. The other isn’t.

Postscript #2: See our related article, 10 Alternatives To Delicious.com Bookmarking.

Postscript #3: See our new post indicating Delicious.com may not be closing down: Not So Fast: Delicious.com May Survive, After All.

BOSS which launched in July 2008 has become a favorite of many developers when building their own search engine off of Yahoo’s data. With the organic Yahoo results being powered by Bing now, and the Microsoft deal closed, Yahoo has decided to switch BOSS to a pay per use model. Microsoft Bing will be powering some of these results, but not all.

They will charge on a cost-per-query model where the price will vary from $0.40 to $0.75 CPM (cost per 1000 BOSS queries). The price, as Yahoo explained, will depend on if you are querying web, image, news or other information. Yahoo said they plan on offering BOSS v1, the free version, for free 60 days after BOSS v2, the paid version, is launched – which is expected in early 2011.

Here are the key points as mentioned by Yahoo:

• BOSS will continue to be a RESTful API for web, image, and news search. Core web and image results will eventually be powered by the Microsoft search platform. In 2011, Yahoo! plans to expand upon these offerings.
• BOSS plans to provide full flexibility to blend, stack, and re-rank results. You know your users best, so we will give you the ability to display exactly what they need. We also aim to continue enhancing the BOSS offering by providing additional vertical search services in the future.
• We will require developers to access BOSS V2 via the Yahoo! Developer Network standard oAuth-based authentication.
• BOSS V2 will provide a self-service payment model. We plan to offer developers the ability to add credit-card authentication at sign-up and to check usage, billing, and other information on a completely redesigned dashboard interface.

It appears that Yahoo is allowed to continue operating BOSS under the Microsoft deal. Some of the results will be powered by Yahoo, while the search results in BOSS will be powered from Microsoft Bing. But Ashim explained that BOSS needs to make financial sense to Yahoo and somewhat implied that they are exploring a fee based structure to use BOSS in the future.

Here is the full post from Ashim:

BOSS Developers,

Thank you for your feedback. We understand your frustration. This process has been long for all of us due to the complex nature of our agreement with Microsoft, and we appreciate your patience.

Under this agreement, Yahoo! is permitted to continue offering the BOSS web service, with search results that would integrate Yahoo! services and content with algorithmic results provided by Microsoft. As always, our intention is to provide a BOSS offering as long as it makes business and economic sense to do so. We are still examining what the BOSS offering will consist of, with some services powered by Microsoft, unique content that Yahoo! currently provides, and the potential for additional Yahoo! content in the future.

Prior to the announcement of the Yahoo!-Microsoft search agreement, we’d already shared our intention to explore a fee-based structure for BOSS. We continue to explore an appropriate fee structure or other revenue model as we work through the future of BOSS.

As you know, we must receive regulatory clearance before actual implementation of the search deal with Microsoft can occur. Only then can we finalize the future shape of BOSS. Of course, we will provide additional clarity and certainty when we can.

Thanks for your attention!

Ashim Chhabra The Yahoo! Search BOSS Team

Hat tip to TechCrunch for spotting this.

We’ve read about what this means for advertisers (more overall traffic from the combined audience, use of Microsoft adCenter for self-serve and Yahoo!’s sales force for premium) and for searchers (they likely won’t notice), but what does the deal means for those who create websites: publishers, web developers, and SEOs?

Web Developers

The hardest hit by this change will likely be developers. Over the last couple of years, Yahoo seems to have shifted its focus from innovating the search index to innovating its developer offerings: encouraging third-party development and creating a “developer ecosystem” for search.

Any developer options that don’t rely on the Yahoo search index may be unscathed. In particular, the non-search development tools and search-related offerings that are solely focused on the user interface may continue to be supported. While Bing will power Yahoo’s search engine, Yahoo will control their user interface and likely will try to continue to differentiate there. That’s means Search Monkey, which enables site owners to enhance how their results appear on Yahoo, is potentially safe.

Build Your Own Search Service (BOSS) likely won’t be so lucky. BOSS is built on the Yahoo index as its foundation. A company can build their own search engine using Yahoo’s underlying technology and differentiate via the user experience.  Essentially, that’s what Yahoo is planning to do now with Bing as their underlying technology. No more Yahoo index likely means no more BOSS. Yahoo all but concedes as much: “We can tell you that BOSS will remain live for the time being.” What does that mean for companies like hakia, OneRiot, Daylife, and Cluuz? And for that matter, all of the developers using BOSS who are now filling the Yahoo BOSS message boards with questions?

On the Yahoo developer blog, Yahoo commented that “For SearchMonkey and BOSS, we currently do not have anything concrete to tell you. Clearly, we’ll need to work with Microsoft to determine what makes the most sense for you and for us.” If BOSS’s future is left up to Microsoft, I have no doubt that future will involve migrating BOSS users to the Bing search API. In order to continue to support BOSS, Microsoft would have to completely recreate it to work with the Bing search infrastructure. Why would they do that when they can increase the audience of a product they already have? It’s possible they’ll add some of the unique BOSS features their search API (such as unlimited queries, ability to mash up the data with other sources, and ability to tweak ranking signals), but I wouldn’t hold my breath. The Yahoo BOSS team is just as in the dark as the developers wanting answers. From a message board post: “What specifically does it mean for BOSS? Honestly the team is still absorbing the implications and we just don’t know.”.

BOSS users could switch to Google’s Custom Search API, but it is more restrictive than Microsoft’s offering, and isn’t really well-suited as the foundation of a search engines or other commercial company. Several other companies offer web indices, such as CommonCrawl and Alexa, so perhaps they or a new company will take advantage Boss’s imminent demise and offer matching features.

Any Yahoo offerings that don’t rely on an underlying index, such as the Yahoo User Interface library are likely going to remain. Yahoo confirmed this in their blog post:

“We’ve also received questions about the future of Yahoo!’s other developer offerings, such as YUI, YQL , and Pipes. We wanted to let you know that today’s news does not affect these products. None of our other non-search developer products are affected.” [Emphasis mine.]

However, look for any search index-based offerings (such as the Maps API and Local API) to be deprecated in favor of the Bing equivalents once the deal goes through.

Search Engine Optimizers and Site Owners

What about those who are concerned with getting customer acquisitions through organic search? How will this change impact them? From a traffic perspective, take a look at how well you’re indexed and ranked in Bing. That’s how well you’ll be indexed and ranked in Yahoo. What do your titles and descriptions look like in the results in Bing? That’s how things will generally look in Yahoo. This might not be a bad thing for site owners, as over the last year, Yahoo’s search quality seems to have been declining to the point that I’ve been wondering if their engineering team had already begun to be phased out or least was spending a lot of time at the bar mourning the likely phase out.

Just as you don’t need to optimize separately for AOL since they use Google’s index, you won’t need to optimize for Yahoo since they’ll use Bing’s index. The exception to this may be in how Yahoo displays results. We’ll have to wait and see exactly what this means, but Bing has been trying to differentiate in display and it supposedly, Yahoo will continue to do that as well. This may mean, for instance:

• SearchMonkey will continue to be important as a way to stand out in the results.
• Hmm. I can’t really think of anything else.

My guess is that the robots-nocontent tag will no longer be supported, since Bing’s infrastructure doesn’t support it. The search engines have already come together to standardize their support of robots.txt and XML Sitemaps, so site owners shouldn’t worry about changing anything with those.

The bigger issue many SEOs are concerned about is Site Explorer. Site Explorer is one of the more reliable tools for competitive backlink research. You can see a substantial list of links to any site, generally ordered according to value. That’s useful stuff! Both Google and Bing Webmaster Tools provide backlink data, but only for your own sites. Yahoo will be unable to maintain Site Explorer without a search index of their own. Will Bing take it over? Well, it could add the feature to its Webmaster Tools, but Microsoft has historically been moving the other direction. They removed the ability to query their index for link data with the link: operator in 2007 and have never brought it back for competitive research.

Microsoft likely won’t be motivated to add a feature that they specifically chose to remove. And it’s not trivial to build the code to query for competitive links and store the data. Believe me, I know. I managed the process for adding non-competitive backlink data to Google Webmaster Tools. As with the potential end to BOSS, the potential end to Site Explorer opens up new opportunities for third-parties. In fact, the same companies who build a web index could provide competitive link data. Currently, SEOmoz  provides Linkscape, which offers some similar features. (Speaking of SEOmoz, Rand Fishkin posted yesterday about the SEO impact of this deal.) Majestic SEO and Exalead provides link data as well.

More generally, will Microsoft step up its efforts with webmaster relationships? Yahoo used to have a fairly significant presence in the community. In addition to Site Explorer, they were a constant at conferences and participated in online discussions. That participation has declined lately, coinciding with the decline in search quality. Microsoft seemed to be rallying with its webmaster relationships with the launch of the Webmaster Center in November 2007. But Microsoft hasn’t updated the Webmaster Center with new features since August 2008. (A minor release in November didn’t add  new features).

Microsoft didn’t respond to my questions about their current and future resource investment in this area. They did recently release a rudimentary SEO Toolkit, although it requires Windows Vista and IIS 7.0 to run.

And what about paid inclusion? Yahoo has long offered Search Submit Pro, which essentially enables sites to pay to be included in the organic listings. Microsoft doesn’t offer a similar product and while it’s certainly possible that Microsoft will add this product to their offerings, paid inclusion is quite a substantial shift in overall approach to organic search. It’s less about the ability to implement the technology and more about belief around what constitutes an “organic” index. Danny Sullivan asked about paid inclusion at the announcement press conference. Carol Bartz, Yahoo CEO replied, ” Paid inclusion, we’ll decide on that later.” But it would be difficult for Yahoo to continue the program on its own, as Yahoo will no longer have control over what pages are included in the search index.

In the end, it’s about the traffic

The big question is will this partnership significantly change market share percentages? Depending on whose numbers you use, Google has either 65% or 74% share in the US (more in some European countries). That puts the combined Yahoo/Microsoft share at 28% or 25.5%. That’s substantial traffic, sure, and worth paying attention to. But what will the share look like in three years once the deal is done and we barely remember Yahoo ever had its own index? My guess is pretty similar to how it looks now. Except Google will probably have slightly higher share. I just don’t see anything game changing here that will cause a mass exodus from the status quo. But I’ve been wrong before. What I do know is that site owners who have ignored how their sites were doing in Bing until now do to low traffic numbers will likely start paying a lot more attention.

Tyler Bell of Yahoo gave the talk at Where about a “geo-enriched location-aware internet.” Where is a very developer-focused event. Bell’s talk was essentially about building the location “infrastructure” that will help support the vision articulated at the Yahoo Search event yesterday. The first aspect of the talk was about  Yahoo GeoPlanet (which provides layers of geographic identifiers and place-based information to developers). Bell announced that in addition to offering an API, GeoPlanet will now allow developers to download all the data for free (with attribution to Yahoo) and host the data themselves.

Yahoo also announced Placemaker. Placemaker is an API that enables developers to pass unstructured data in documents to the service. It extracts the location information in those documents and allows the information to be associated with one or more locations (publishers/developers need to decide which places are most important). MetaCarta offers a similar capability.

Absent my attendance at the Yahoo Search event yesterday these would have seemed fairly map-geeky announcements. But in the context of Yahoo’s larger project to more directly tie search and the internet to the physical world this was quite interesting.

Here’s more information from the Yahoo Geo Blog.

The new tools include:

• Integration with Delicious content – end-users can get Delicious saves, top tags, and count information within their search results
• Advanced language capabilities – developers can now filter specific language results for a given market. BOSS now supports Czech, Hungarian, and traditional Chinese
• New sorting functionality – BOSS News service can now sort by date and/or a specified time range in days, weeks, or hours

Yahoo’s announcement has more information about all three tools, including a few code examples.

Fee structure

The BOSS charges kick in only once queries exceed 10,000 a day. After that, there’s a tiered model based on what is requested through the API. The pricing will go into effect next quarter.

New features

In exchange for the introduction of fees, BOSS has added several new features. Key among them is the ability to monetize results. Yahoo! has adjusted the terms of service to allow developers to use third-party monetization platforms in conjunction with BOSS results. This is a shift from Yahoo!’s initial plan, which was to provide Yahoo-powered ads with a revenue sharing model. When BOSS first launched, Yahoo told me that they were planning to require Yahoo ads at some point and that “over the next several months, a BOSS monetization capability, using Yahoo! search advertising and potentially other models, will be made available for partners and developers to create a search revenue stream for their business.”

In addition, Yahoo! is expanding the maximum number of results returned from 50 to 1,000 and will be introducing an SLA to ensure reliability.

BOSS will be offering access to SearchMonkey data as well, starting with structured markup that the Yahoo! web crawler has extracted. This is yet another move to encourage the use of semantic standards on the web. (In a session at SMX West yesterday, Arnab Bhattacharjee, Senior Director, Yahoo! Search Technology and Engineering at Yahoo! reiterated Yahoo!’s encouragement of the use of structured markup.)

They’re also making longer abstracts of documents available (up to 300 words from the previous 170).

New functionality for webmasters: link data

The most interesting  new feature for webmasters may be the ability to query Site Explorer link data through the BOSS API. Presumably, however, the number of results returned will be limited to 1,000,  which may not completely satisy those looking for backlink data.

While the BOSS improvements are encouraging, it remains to be seen if the new pricing structure slows down Yahoo!’s efforts to disrupt the market by jump starting those with little cash.

BOSS is Yahoo’s Build Your Own Search Service product, which lets anyone take Yahoo’s data and create their own search engine. It’s grown into part of Yahoo’s “open strategy” where it hopes to make gains on the web by supposedly pulling in data from everywhere as well as being open to others using its data.

TweetNews was trotted out by Yahoo as the latest example of how BOSS is succeeding. It takes stories showing up on Yahoo News and ranks them based on how much Twitter activity they’re receiving.

Kind of cool, right? Sure, it is. But then again, TweetNews was created by Vik Singh, a Yahoo programmer who works on the BOSS team. Why not just build this as Yahoo product itself? Hosted on a Yahoo site, rather than depending on Google’s App Engine (and dying when it exceeded the allowed quota). Moreover, why not build Yahoo’s own version of Twitter search?

People can argue over whether Twitter has mass appeal or not. You cannot argue that at this point, news DOES break out on Twitter far ahead of any other source. I watched firsthand how news about fires breaking out around me in Southern California last November was reported on Twitter before it hit Google News and other sources. The same is true for two different earthquakes I’ve gone through in the past six months. There are any number of other examples where Twitter has news that’s coming out and being eventually cited. Yesterday’s New York plane crash on the Hudson River was yet another case in point.

Twitter has powerful, compelling material. And if you want to find that material, you go to Twitter. Not to Yahoo. Not to Google, the king of search. Twitter.

I think it’s a missed opportunity for both Google and Yahoo. And for Yahoo in particular, while Google is closing down some search projects (arguably for good reasons), Yahoo should still be building its own unique and valuable search tools — not depending so much on others to do it.