Google: News Sites May Have Seen A Decline In GoogleBot Crawling In March
Google’s Matt Cutts tweeted that news related sites, such as this one, may have noticed a drop in the crawl statistics within Google Webmaster Tools.
If you did notice that drop, Matt says you do not need to worry. Google News added if-modified-since support in March, which would be the reason for the crawl changes. Matt added that “Google News doing smarter crawling now” and thus being more efficient.
I’ve checked the crawl stats for seroundtable.com, a Google News publisher, and indeed, the crawl stats have declined significantly in March 2014:
To learn more about how GoogleBot handles the if-modified-since tag, see this blog post.
Here is Matt’s set of tweets:
If you have a news site and it looks like Google is crawling less since ~3/7, don’t worry. Google News added if-modified-since support then.
— Matt Cutts (@mattcutts) April 8, 2014
So it’s just Google News doing smarter crawling now. More info about if-modified-since: https://t.co/1TrtFVnypD
— Matt Cutts (@mattcutts) April 8, 2014
Explore more on
Add Search Engine Land as a preferred source on GoogleEditorial disclosure
Search Engine Land is owned by Semrush. We remain committed to providing high-quality coverage of marketing topics. Unless otherwise noted, this page’s content was written by either an employee or a paid contractor of Semrush Inc.
About the author
