Googlebot will crawl and index the first 15MB of content per page

Make sure the content you want Googlebot to crawl, index and ultimately rank is within the first 15MB of that page.

Chat with SearchBot

Google has updated its help document on Googlebot to specify that Googlebot will crawl up to the first 15MB of the page and then stop. So if you want to ensure that Google ranks your page appropriately, make sure Googlebot can crawl and index that part of the page within the first 15MB.

What is new. In the Googlebot help document, Google added this section that reads:

Googlebot can crawl the first 15MB of content in an HTML file or supported text-based file. After the first 15MB of the file, Googlebot stops crawling and only considers the first 15MB of content for indexing.

Why we care. In general, you probably want to keep your pages pretty light for both users and search engine crawlers. But here Google is being very clear about how much Googlebot will consume from your page.

A good way to test this is to use the URL Inspection tool in Google Search Console and see what parts of the page Google renders and sees within the debugging tool.

Postscript. Google added more clarification about this change over here.


About the author

Barry Schwartz
Staff
Barry Schwartz is a technologist and a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics.

In 2019, Barry was awarded the Outstanding Community Services Award from Search Engine Land, in 2018 he was awarded the US Search Awards the "US Search Personality Of The Year," you can learn more over here and in 2023 he was listed as a top 50 most influential PPCer by Marketing O'Clock.

Barry can be followed on X here and you can learn more about Barry Schwartz over here or on his personal site.

Get the newsletter search marketers rely on.