Duplicate Content: The Biggest Challenge To Successful International SEO

Duplicate content is the number one trap for newbies to international SEO in their efforts to roll out a site to all parts of the globe. Ghoulish in character, it lays in wait ready to jump the unsuspecting. It isn’t really a “language” issue—but language is the siren which tempts the unwary into the trap.

If you’re running a successful US English-language site and you want to replicate your business model in other markets, what is the first thing you do? Localize the site in a different language? Of course you don’t. The first thing you do is you try another English language market. If at the same time you add other languages, which do you choose—Polish? Probably not. You generally choose a language such as Spanish, French or German—globally important languages with many speakers and spoken in many countries. Mostly, you avoid “language” altogether and stick with English!

When you localize a web site into a single country language, such as Polish which is principally only spoken in Poland, or Hungarian, which is principally only spoken in Hungary, you significantly reduce the likelihood of a duplication problem to nil—even if the content you started with and the content you publish are the same. So, a single-nation language means no duplication problem.

SEO in the US and UK is the same—and different

Many of the not-yet-international US sites that select a new market to target seem to choose the UK. This makes sense, of course. Despite what George Bernard Shaw said (“England and America are two countries separated by a common language,”) the UK is closer in culture and language to the US than many others (except of course that we drive on the correct side of the road). It is also wise to enter a market via a nation that you pretty much understand (sic). You would also expect SEO requirements for the UK to be pretty much the same as the US—apart from having to change some “z’s” to “s” of course.

Quite rightly marketers and chief executives say to themselves, “Whatever happens with our export test, at least things are running OK at home so the risk is not too great.” Sadly, the catch with choosing the UK as your first new market is that if things do go wrong, it could well be on your domestic US site that the troubles emerge.

The blame game starts

And duplication issues are not so easy to spot. The big problem with duplication issues is that they are not necessarily huge in impact—they just hold you back and you start to look for all sorts of possible causes. Performance just isn’t what you expected, and everyone starts looking for someone—or more usually—something to blame.

Here’s a recent client conversation that illustrates my point.

Client: “Things were going great but in October we just didn’t see the traffic growth we were expecting—in fact it was lower than our direct access growth.”

Andy: “Did you make any changes in October?”

Client: “No nothing at all. Oh apart from we launched the Irish site, of course.”

Andy: “And how similar is the Irish site to your US site?”

Client: “It’s very different as it’s very small—we don’t offer as much to Ireland, so that can’t be it, right?”

Andy: “The home page?” Client: “Sure the home page is the almost the same as the US but….”

In the wrong place at the wrong time!

It would be true to say that many people put sites into pigeon holes and only consider the problems of the Irish site in Ireland or the UK site in the UK. The connections between sites is not where most folk look first—but they should.

Don’t forget also, that frequently the issue with duplication is not that the site doesn’t rank—but that the “wrong” pages rank in the wrong place. The classic example is that you’re trying to sell something (say downloadable software) to Brits in dollars because the top ranking page for your key search term is from the US. Now Brits don’t think money is actually money unless Her Majesty The Queen is present—so dollars just don’t sell as well and the great British Pound page, which would sell, is down on page three or four of the search engine results pages—or not showing at all.

Using “modern” solutions

One thing you cannot do is simply shuffle paragraphs around on the page to “deduplicate” things. Nope, Google is wise to that, deeming each significant block of content on a site a “shingle,” and hides those pages behind that “Some pages were omitted” link at the end of results. A recent case of a translation agency with multiple sites and a duplication problem boiled down largely to a couple of paragraphs—they just happened to feature on all the home pages of their nearly 20 sites.

More modern solutions are worth looking at to help. You should, of course, adopt local country domains to assist the search engines identify which site belongs where. You can plug into Webmaster Central and tell it where to stick your pages and to help with things there’s now that cross-domain canonical tag available. All useful in extremis. I’d suggest though that you get under the hood and look at fixing the root causes of your duplication problem first, rather than relying on these workarounds. Otherwise the next time you come to make changes someone will forget the canonical tag or change the URL name or something and that will put you back exactly where you were before. Duplicated.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: SEO | Multinational Search


About The Author: is a linguist who has been specializing in international search since 1997 and is the CEO of WebCertain, the multilingual search agency and Editor-in-Chief of the blog Multilingual-Search.com. You can follow him on Twitter here @andyatkinskruge.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://www.seansupplee.com SeanSupplee

    I was really glad to see this article come up here today. I go into a lot of forums and I always see duplicate content questions about SEO all the time. There are some on the side that say its not an issue and then others that say it is. Honestly it is an issue and you have to create unique content. If you did not the entire web would be the same article over and over again.

    Glad to see this addressed in the right spotlight and even out of a context in which I was thinking about it when creating a UK or another language version of your site.

  • tatiana_london

    We use the same root domain (I know it’s not perfect) and reproduce our site structure in 2-3 languages now, is that a problem if we have duplicate or almost duplicate urls?

  • http://www.bealoud.com BeAloud

    Having almost duplicate urls wouldn’t be a problem, but if your website isn’t too big, I would use localised keywords in urls (the same you would use in the content: us.shop.com/flowers, fr.shop.com/fleurs, it.shop.com/fiori and so on).

    Even if you are just using different folders, like /fr and /de, you still can use multiple XML Sitemaps to target different countries and languages more effectively.

  • http://www.searchkingdom.co.uk RobAndrews

    Wow, a big issue and well done Andy for highlighting some of the dup content traps in geographical/language site expansions.

    In my experience every single situation is different and each expansion should be planned, whilst taking into account all the individual factors.

    There are a host of options available (subdomains/folders/top level domains/IP delivery/etc.). You have to find the right mix for your environment and goals.

    Really, really important to get good advice here and not ‘wing it’.


Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide