The Yahoo Blog issued a weather report for changes to rankings in Yahoo Search, along with news that they are now supporting the X-Robots-Tag directive — a way to control indexing of content that cannot accept meta robots tags.

Google also supports X-Robots, which gives webmasters the ability to define robots.txt like rules within http headers, as opposed to just the META data within HTML pages.

Yahoo provided a few examples of how it can work:

• X-Robots-Tag: NOINDEX — If you don’t want to show the URL in the Yahoo! Search results. Note: We’ll still need to crawl the page to see and apply the tag, so if you don’t wish to have the page crawled, use robots disallow on robots.txt.
• X-Robots-Tag: NOARCHIVE — If you don’t want to display the cache link in the search results page.
• X-Robots-Tag: NOSNIPPET — If you don’t want to display a summary in the search results page.
• X-Robots-Tag: NOFOLLOW — If you don’t want Yahoo! to crawl links in the page.

Related Topics: SEO: Blocking Spiders | Yahoo: Search

About The Author: Barry Schwartz is Search Engine Land's News Editor and owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry's personal blog is named Cartoon Barry and he can be followed on Twitter here. For more background information on Barry, see his full bio over here. See more articles by Barry Schwartz