Google: News Sites May Have Seen A Decline In GoogleBot Crawling In March

Google’s Matt Cutts tweeted that news related sites, such as this one, may have noticed a drop in the crawl statistics within Google Webmaster Tools. If you did notice that drop, Matt says you do not need to worry. Google News added if-modified-since support in March, which would be the reason for the crawl changes. […]

Chat with SearchBot

newsGoogle’s Matt Cutts tweeted that news related sites, such as this one, may have noticed a drop in the crawl statistics within Google Webmaster Tools.

If you did notice that drop, Matt says you do not need to worry. Google News added if-modified-since support in March, which would be the reason for the crawl changes. Matt added that “Google News doing smarter crawling now” and thus being more efficient.

I’ve checked the crawl stats for seroundtable.com, a Google News publisher, and indeed, the crawl stats have declined significantly in March 2014:

google-news-crawl-stats-drop

To learn more about how GoogleBot handles the if-modified-since tag, see this blog post.

Here is Matt’s set of tweets:

 

 


About the author

Barry Schwartz
Staff
Barry Schwartz is a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on Twitter here.

Get the must-read newsletter for search marketers.