The Google Webmaster Central blog announced Google has published a new set of documents in the Google Code section on how to control Google from crawling and indexing your site. You can read the set of documents over here.

The technical documents are broken down into five section:

• Getting Started
• Robots.txt specification
• Robots meta tag and X-Robots-Tag specification
• Google’s crawlers
• References

I personally printed them out as my weekend reading.

Related Topics: Channel: SEO | Google: SEO | SEO: Blocking Spiders

About The Author: Barry Schwartz is Search Engine Land's News Editor and owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry's personal blog is named Cartoon Barry and he can be followed on Twitter here. For more background information on Barry, see his full bio over here. See more articles by Barry Schwartz

Connect with the author via: Email | Twitter | Google+ | LinkedIn