Google Releases Details On Controlling GoogleBot & Google’s Crawl

The Google Webmaster Central blog announced Google has published a new set of documents in the Google Code section on how to control Google from crawling and indexing your site. You can read the set of documents over here. The technical documents are broken down into five section: Getting Started Robots.txt specification Robots meta tag […]

Chat with SearchBot

The Google Webmaster Central blog announced Google has published a new set of documents in the Google Code section on how to control Google from crawling and indexing your site. You can read the set of documents over here.

The technical documents are broken down into five section:

I personally printed them out as my weekend reading.


About the author

Barry Schwartz
Staff
Barry Schwartz is a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on Twitter here.

Get the must-read newsletter for search marketers.