Google changed the way it works, and no one really noticed

In October 2017, Google announced it had updated the way it labeled country services on the mobile web, the Google app for iOS and desktop Search and Maps.

From Google:

Today, we’ve updated the way we label country services on the mobile web, the Google app for iOS, and desktop Search and Maps. Now the choice of country service will no longer be indicated by domain. Instead, by default, you’ll be served the country service that corresponds to your location. So if you live in Australia, you’ll automatically receive the country service for Australia, but when you travel to New Zealand, your results will switch automatically to the country service for New Zealand. Upon return to Australia, you will seamlessly revert back to the Australian country service.

At the time, it seemed this update was more of a minor change or inconvenience when trying to check international results. Instead of going to the version of Google for a particular country, the results were now based on your location or the location in your Google settings.

Google stated:

This update will help ensure that you get the most relevant results based on your location.

This seemingly minor change actually had a huge impact for websites operating in multiple markets.

Before the October update, if pages were duplicate or near-duplicate, they would be folded together in Google’s index, and only one version would show as it was selected as the canonical version. Often it would be the wrong language or country version of a page that was shown to users.

For example, if your pages were the same in the US and Australia, people searching in Australia might see the US version of the page. This wasn’t a problem with translated language versions, as they were not seen as duplicate. Now, Google is trying to show the best version of the page, even if they are folded together, by picking the version that best matches the users and respecting hreflang tags.

Before the change

Before this update, Google representatives said if pages were folded together because of duplicate content, they would not see the return links for the hreflang tags and that Google was folding the content for you. This was supposed to make your website “stronger” by having fewer but “stronger” pages.

In January 2016, Google Webmaster Trends Analyst John Mueller was replying to a thread in the webmaster help forum talking about this issue.

What’s happening here is that we’re getting a bit confused with all the duplicate URLs that you have on your site. For every country, you seem to have the same content. We’re taking some of these duplicates, and folding them into a single URL to make things easier. Because of that, we don’t see the return links from those duplicates. In other words, for the URLs that you do have indexed, we’re doing the right thing, and applying your hreflang markup there. However, all the unnecessary duplicates end up causing these warnings in Search Console. My recommendation would be to only include unique & relevant content within your site — only include countries that you really have something unique for. This helps our algorithms (we don’t have to filter out duplicates & pick one of the URLs for you), and helps you too (reduces the bloat in your website, it makes the remaining pages a bit “stronger”, and makes it easier to diagnose technical issues like these).

John talked about this again in a Webmaster Central office-hours hangout on November 2016.

From the 1:27 mark:

We think these pages are either identical or so similar that we can fold them together… hreflang is a really strong signal for us, but if we see even stronger that this is essentially the same website, then we think that it makes more sense for you, the users, that we fold them together so that you have one stronger website rather than two websites that are essentially identical that aren’t as strong individually.

As Google crawls and sees multiple versions of the same page, it determines it only wants one main version for its index and folds the others together. Here is how John Mueller described in a recent Reddit AMA how Google indexes just one version and folds the other into the same database record:

It’s really a duplicate content issue, and the pages are folded together in Google’s index, so it broke the hreflang tags because it can’t make the connections between certain pairs. How can an Australian page be an alternative to the UK page when the Australian page is folded and indexed as the US page?

There were a few commands to troubleshoot this at the time. If site:domain/page searches or searches for a keyword showed a wrong version, that was a good indication.

Info:domain/page would tell you the canonicalized version or the version the page was indexed as. If it showed a different page, then the folding was happening, and you’d probably end up with the wrong version showing in the SERPs.

You could also check the cache of the page in the SERPs or with the cache:domain/page command. One caveat on this was that the site may have some auto-redirecting logic based on internet protocol (IP) or browser language, and cache could have been inaccurate.

It came down to the pages being duplicate content, not that they had their hreflang tags wrong.

Here’s an example, check out info:https://google.com.au and you’ll see the Canadian version (ca).

It’s the same with the cache; cache:google.com.au would show the cache for the Canadian version of the page and indicate they were folded together, and the Canadian version became the main version.

At the time, people looking for Google Australia would likely have seen Google Canada. Then came the update that changed everything.

Changes since the update

The messaging from Google has changed since the update. Google is saying that signals are consolidated, and even if pages are folded together, hreflang tags will be respected, and the right version will show. Many of the same results that showed the incorrect versions before are now showing the correct local version.

Listen to what Google Webmaster Trends Analyst Gary Illyes said during an interview in November 2017 (after the update):

With hreflang, what you are doing is telling that pages in a cluster (so basically A links to B, B links to A) that those pages should be able to use each others ranking signals.

John Mueller, in another Webmaster Central office-hours hangout in March 2018, said:

We can follow the hreflang links as well, even in cases where we pick one version as a canonical. This is really common in cases where the same language content is used across multiple countries, then what we might say is we understand the hreflang links between these different versions and we understand that the content is actually the same so we pick one version of the content to index but we swap out the URLs anyway because of the hreflang connection that we have … What also sometimes happens is that we understand language or country based on other factors like URL, internal linking, different language or country links, then sometimes we can pick up those connections even without clear hreflang guidance.

We still have the same issues as before, with the info: and cache: commands showing the Canadian version of the page because they were folded together, but now, depending on your location or your Google search settings, you will see the correct version of the page, like in this search with my settings as Australia:

Even though the pages are folded together, where the hreflang tags would be broken before and the wrong page would be shown, now it all works as it should, and you see the version of the page that is right for you based on your location or settings.

In addition, where they were folding pages together before to make a stronger version of the page, now it seems they are consolidating signals to the set of pages. One new issue this seems to have caused is when you are looking for a particular version of a page, like searching for “Google Australia” while in a different country, you may end up with the canonicalized version showing.

So when I look for Google Australia, what I find is google.ca instead.

Conclusion

I’m not sure I fully trust the way this works now, and obviously, Google still has some work to do when searching for individual versions, but the change has solved a lot of headaches for SEOs.

I would recommend that you continue to localize the different versions of the pages and to differentiate them as much as possible, as this will provide a better experience to local users. This change has made it less likely for pages to show a different language or country version in the search results, and for that, I am happy Google made the update.