Google Releases Details On Controlling GoogleBot & Google’s Crawl
The Google Webmaster Central blog announced Google has published a new set of documents in the Google Code section on how to control Google from crawling and indexing your site. You can read the set of documents over here.
The technical documents are broken down into five section:
- Getting Started
- Robots.txt specification
- Robots meta tag and X-Robots-Tag specification
- Google’s crawlers
I personally printed them out as my weekend reading.
Some opinions expressed in this article may be those of a guest author and not necessarily Search Engine Land. Staff authors are listed here.
(Some images used under license from Shutterstock.com.)
Everything you need to know about SEO, delivered every Thursday.