Google Search Descriptions Now Say If GoogleBot Is Blocked
When you block GoogleBot, Google’s web crawler, the page that is blocked can still show up in the Google search results because of links pointing to the page. When that happens, the search listing looks awkward, because Google isn’t able to crawl the page and make a title and description. Instead, Google will make a best guess at the page title and often leaves the description (search snippet) blank.
Now, instead of showing a blank search snippet, Google is showing a message that communicates why Google cannot show a search snippet.
The message reads:
A description for this result is not available because of this site’s robots.txt – learn more.
This clearly describes that Google is now able to show a description for the page because something is preventing GoogleBot from crawling it. The learn more link takes you to a help document with more details on why this is the case.
Not only does this give the searcher a better search experience, it also provides a webmaster with another notification of site issues outside of Google Webmaster Tools.
Explore more on
Editorial disclosure
Search Engine Land is owned by Semrush. We remain committed to providing high-quality coverage of marketing topics. Unless otherwise noted, this page’s content was written by either an employee or a paid contractor of Semrush Inc.
About the author
