Yahoo Search Weather Update & Support For X-Robots Tag

The Yahoo Blog issued a weather report for changes to rankings in Yahoo Search, along with news that they are now supporting the X-Robots-Tag directive — a way to control indexing of content that cannot accept meta robots tags. Google also supports X-Robots, which gives webmasters the ability to define robots.txt like rules within http […]

Chat with SearchBot

The Yahoo Blog issued a weather report for changes to rankings in Yahoo Search, along with news that they are now supporting the X-Robots-Tag directive — a way to control indexing of content that cannot accept meta robots tags.


Google also supports X-Robots, which gives webmasters the ability to define robots.txt like rules within http headers, as opposed to just the META data within HTML pages.

Yahoo provided a few examples of how it can work:

  • X-Robots-Tag: NOINDEX — If you don’t want to show the URL in the Yahoo! Search results. Note: We’ll still need to crawl the page to see and apply the tag, so if you don’t wish to have the page crawled, use robots disallow on robots.txt.
  • X-Robots-Tag: NOARCHIVE — If you don’t want to display the cache link in the search results page.
  • X-Robots-Tag: NOSNIPPET — If you don’t want to display a summary in the search results page.
  • X-Robots-Tag: NOFOLLOW — If you don’t want Yahoo! to crawl links in the page.

About the author

Barry Schwartz
Staff
Barry Schwartz is a technologist and a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics.

In 2019, Barry was awarded the Outstanding Community Services Award from Search Engine Land, in 2018 he was awarded the US Search Awards the "US Search Personality Of The Year," you can learn more over here and in 2023 he was listed as a top 50 most influential PPCer by Marketing O'Clock.

Barry can be followed on X here and you can learn more about Barry Schwartz over here or on her personal site.

Get the must-read newsletter for search marketers.