Google launches new crawl stats report in Search Console

Google has announced the launch of a new crawl stats report in Google Search Console.

You may be able to access this report over here but keep in mind Google is rolling this out now and it might generate an error for you.

What’s new. Google said the new report makes it easier for developers, webmasters, SEOs, etc to find issues with Google crawling. What is new is:

1. Total number of requests grouped by response code, crawled file type, crawl purpose, and Googlebot type.
2. Detailed information on host status
3. URL examples to show where in your site requests occurred
4. Comprehensive summary for properties with multiple hosts and support for domain properties

New crawl stats report. This new chart shows your crawl stats over time, showing you the total crawl requests over a time period or by day, total downloaded data and the average response time.

Google also has a way to break it down into “grouped crawl data.” The report provides data on crawl requests broken down by these groups: (1) response, (2) file type of the fetched URL, (3) purpose of the crawl request, (4) and Googlebot agent.

Hosting issues are also uncovered with this report. So Google can show you why it had issues accessing your site. Google said “host status details in the report let you check your site’s general availability to Google in the last 90 days.” And for domain properties with multiple hosts, you can check the host status for each of the top hosts presented in the report summary view.

Why we care. Google has supported the legacy crawl stats report for years and with the new Google Search Console here, Google has not yet migrated this feature. Not only has it really improved on the reports, it has given us actionable and useful data to use when debugging crawling issues. To learn more about this report, see this help document.

About The Author

Barry Schwartz

Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry's personal blog is named Cartoon Barry and he can be followed on Twitter here.