Google to update GoogleBot’s user agent

Google announced that starting in December Google will update Google’s user agent to represent the version of Chrome that GoogleBot is running. It is important to update any code you may have that may look at the user agent of GoogleBot to support this change going forward.

New user agent. When Google launched the Evergreen GoogleBot, Google intentionally kept the old user agent. This was done so that anyone who may have hard-coded any detection methods for GoogleBot would not have any issues with the new Evergreen GoogleBot.

The new user agent will soon change to show the current version of Chrome that GoogleBot is using when crawling your web site.

Before. Now this is what GoogleBot’s user agent looks like on mobile and desktop:

Mobile:
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Desktop:
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

OR

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36

After: This is how GoogleBot’s user agent will look like after this change in December. In the following user agent strings, “W.X.Y.Z” will be substituted with the Chrome version Google is using. For example, instead of W.X.Y.Z you’ll see something similar to “76.0.3809.100”:

Mobile:
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Desktop:
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

OR

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36

What to look out for. If you have any user agent detection scripts that are hard coded for GoogleBot, you need to make sure to support this new user agent pattern. Google said, “We’ve run an evaluation, so are confident that most websites will not be affected by the change.” “If your site looks for a specific user agent, it may be affected. You should use feature detection instead of user agent sniffing. If you cannot use feature detection and need to detect Googlebot via the user agent, then look for “Googlebot” within the user agent,” Google added.

How to test it. Here is how to test to make sure your pages support this after you make the change to your code or while in development mode. You can override your user agent in Chrome using these instructions.

Why we care. GoogleBot is getting a new user agent, and that is exciting. But it can impact your site if you had any user agent detection methods for GoogleBot. Make sure to test your site to see if it supports the new user agent. Most sites probably do not need to worry about this but you have done any advice GoogleBot detection, you may need to take steps to update those scripts.

About The Author

Barry Schwartz

Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry's personal blog is named Cartoon Barry and he can be followed on Twitter here.