Yahoo Site Explorer: What’s The Status Of the Link Data?

Earlier this week, site owners noticed that substantial Yahoo Site Explorer link data seemed to be missing. A glitch? Gone for good? The first signs of integration into Bing webmaster tools? I talked to Yahoo and got at least some initial answers. Many site owners regularly use Site Explorer for information on external links to their own sites and competitive sites and since the announcement that Yahoo search results would be powered by Bing, many have wondered if Yahoo would continue to make this data available. On the Yahoo Site Explorer suggestion board, a message from Site Explorer product manager Hemant Minocha seemed to indicate that Yahoo was working with Bing to get updated data.

We are working with the Microsoft team to ensure that webmasters can continue to use Yahoo! Site Explorer to view inlinks (a.k.a backlinks) to a given webpage or a website and to further be able to export them as CSV if desired. Please note that this functionality will be available only through the Site Explorer UI and not through the Webservices API.

This message aligns with a Yahoo blog post on August 17th that indicated:

The Yahoo! Site Explorer team is planning tighter integration between Site Explorer and Bing Webmaster Center to make the transition as smooth as possible for webmasters…. To keep things simple, we will share site information you provide on Site Explorer with Microsoft during this transition period. When Microsoft fully powers the Yahoo! Search back-end globally, expected in 2012, it will be important for webmasters to use Bing Webmaster Center as well. The Bing tool will manage site, webpage and feed submissions. Yahoo! Site Explorer will shift to focus on new features for webmasters that provide richer analysis of the organic search traffic you get from the Yahoo! network and our partner sites.

But according to my conversion with Yahoo, the data that’s missing now is unrelated to the Yahoo/Bing integration. Yahoo told me, “Yahoo! is investigating this report and will work to resolve it quickly if an issue is found, however it is unrelated to the longer term question we previously addressed regarding future support for backlinks.” Yahoo reiterated that the long term plan is to continue to provide external link data via Site Explorer.

Where Is the Link Data Coming From?

Since Bing is powering Yahoo’s index, is the external link data provided by Site Explorer stale? “Not at all”, says Yahoo. “Link data in Site Explorer is coming from Yahoo’s data today. We continue to maintain Yahoo!’s crawlers and indexers during this transitional period, ensuring fresh data in Yahoo! Site Explorer.  The US and Canada are currently the only markets for which certain back-end functions of Yahoo! Search are being powered by the Microsoft search platform.” However, it does sound like Yahoo will transition to Bing data at some point: “We are working with Microsoft to ensure that we will continue to have comprehensive, fresh link data for Yahoo! Site Explorer when we switch from Yahoo!’s data to Microsoft data in the future.”

Should Site Owners Continue To Provide Data to Site Explorer?

Site Explorer does more than provide external link data. It also enables site owners to provide data to Yahoo. You can submit XML Sitemaps and provide details on how to handle URL parameters, remove URLs from the index. You can also view query data, crawling and indexing stats, crawl errors, and Delicious activity. For now, Yahoo is stressing that (until 2012) all non-US and Canada searches are still powered by the Yahoo index (except, of course, Yahoo Japan, which is now powered by Google), and so if you care about international Yahoo search traffic, you should continue to provide input data (such as XML Sitemaps and URL parameters) to Yahoo. The Delicious data should continue to be accurate, of course. The robots-nocontent tag is likely also only valuable for Yahoo’s international properties, as Microsoft previously decided against supporting it. Does the URL removal functionality still work to remove results from the Bing-powered, Yahoo-displayed index? Are the query stats still accurate for Yahoo searches? Yahoo hasn’t yet commented on these questions or on the value of the indexing stats and key terms (are these stats for Yahoo’s international indices?). And what about crawl errors? The sites I own show “no crawl errors”. Is this because they don’t have any errors or because Yahoo is no longer reporting them? I’ll update as I learn more.

Yahoo Slurp Continues to Crawl the Web

If you review your server logs regularly, you likely have already noticed that Yahoo Slurp continues to crawl the web.  It sounds like this will continue, both for the purposes of Yahoo’s international search properties and for Site Explorer data until early 2012.

Where Else Is External Link Data Available?

Google webmaster tools provides link data to verified site owners (and recently revamped those reports). When Bing reintroduced their webmaster tools, they removed link data, but at SMX East a few weeks ago, they announced they would be providing new link data, although this has yet to launch. For data on competitive links, several data sources exist, including:

• SEOmoz Linkscape
• Majestic SEO
• Exalead link: operator
• SearchDNA
• Link Diagnosis