Yahoo Search Weather Update & Support For X-Robots Tag

The Yahoo Blog issued a weather report for changes to rankings in Yahoo Search, along with news that they are now supporting the X-Robots-Tag directive — a way to control indexing of content that cannot accept meta robots tags.

Google also supports X-Robots, which gives webmasters the ability to define robots.txt like rules within http headers, as opposed to just the META data within HTML pages.

Yahoo provided a few examples of how it can work:

  • X-Robots-Tag: NOINDEX — If you don’t want to show the URL in the Yahoo! Search results. Note: We’ll still need to crawl the page to see and apply the tag, so if you don’t wish to have the page crawled, use robots disallow on robots.txt.
  • X-Robots-Tag: NOARCHIVE — If you don’t want to display the cache link in the search results page.
  • X-Robots-Tag: NOSNIPPET — If you don’t want to display a summary in the search results page.
  • X-Robots-Tag: NOFOLLOW — If you don’t want Yahoo! to crawl links in the page.

Related Topics: Channel: SEO | SEO: Blocking Spiders | Yahoo: Search


About The Author: is Search Engine Land's News Editor and owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry's personal blog is named Cartoon Barry and he can be followed on Twitter here. For more background information on Barry, see his full bio over here.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.

Comments are closed.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide