Don’t Penalize Yourself: Mobile Sites Are Not Duplicate Content

One of the great things about SEO is the almost scientific way the community approaches the unknown to discover something true that they can apply to their sites. This applied to desktop SEO more than ever in the early 2000s, prior to SEOMoz’s first ranking factors study, when pioneers in the field experimented with their web properties to see which changes would drive the most traffic, and eventually conversions.

While this still happens in desktop SEO, I think it happens more now in areas like video, mobile and social SEO, where SEOs are still trying to discover how non-web search behavior and search algorithms change—if they change at all—the practice of building content to drive the most users from search engines. Knowing this, it’s understandable that the myth of mobile sites as duplicate content has persisted the way it has, in spite of the evidence to the contrary.

For those unfamiliar with the concept, mobile SEO experts, who are often experts in desktop SEO who apply the same concepts to the mobile web, often say that mobile web sites should be considered duplicate content and should be blocked from being indexed by the desktop bot, while desktop versions of sites should be blocked from being indexed by the mobile bot. As with duplicate content recommendations in general, the gist is that web site owners need to indicate which site is relevant to which bot to avoid having the wrong content suppressed when someone does a relevant search, or to avoid splitting your link popularity on what is essentially a duplicate version of your canonical page.

Makes sense in theory. Here’s where it falls apart.

If you enter the query [facebook mobile] on the desktop version of Google, on something other than a mobile device, you get as the first result. The same thing happens when you look for the mobile version of the other nine of Nielsen’s top ten mobile web sites of 2009. In every case both the mobile and desktop versions of each web site are indexed in the “web” index, and they’re returned when a user enters relevant queries. They’re not in the supplemental results, and they’re not suppressed as duplicate content. They’re mobile sites, and they’re returned for relevant queries.

mobile indexing info nielsen top sites

This phenomenon has been in play for years, so I think the idea that mobile sites are duplicate content is really more myth than reality. It’s also unlikely, in my opinion, to change anytime soon, because the search engines, whose modus operandi is to return relevant content for relevant queries, get desktop and smartphone queries for mobile content. If you consider some of the top navigational queries for the top ten Nielsen mobile sites of 2009, you can see this phenomenon in action.

desktop v mobile queries

The query [facebook mobile], to use the same example, gets about 368,000 queries per month from users of desktops (and presumably smartphones) according to Google keyword data. If Google were to show the desktop site for this query just because the user was querying the search engine that used the desktop index, Google would not be showing relevant content for the query, and would not last very long as a search engine that people continued to find useful. This is really a key difference from most types of desktop duplicate content, where a user is unlikely to request the variations of content that Google and other engines classify as duplicates. Can you imagine someone searching for a specific session id page or a non-www version of a page? Let alone 368,000 of them per month? If the search engines relegated desktop content to the desktop index and mobile sites to the mobile index, they would not be able to return relevant content for relevant queries.

Knowing that search engines index and return mobile content for relevant queries, and that they have blended mobile ranking algorithms to rank mobile content for mobile queries and desktop content for desktop queries, it’s clear to me that they don’t treat mobile sites like traditional desktop duplicate content. Knowing this, it’s a wonder that many mobile SEO experts, including some whose opinions I generally respect, would continue to recommend blocking your mobile site from Googlebot and other “web” crawlers.

If you want maximum visibility for your mobile content, and to be truly optimized for mobile search, the best practice is to make it clear to the engines that you have a mobile site, optimize it for mobile queries and make it essential to mobile users, and then trust the engines to return it when they think it’s relevant. Don’t block the mobile site from being returned in desktop or smartphone results, as it’s relevant to more than just the search engine’s mobile index.

To be fair, the search engines are in part to blame for this oft-recommended worst practice, as they have at one point called a certain type of mobile content duplicate content. Prior to the iPhone’s release, back when mobile content was something very different than it can be today, the Google webmaster Central blog called out “forums that generate both regular and stripped-down mobile-targeted pages” as duplicate content. In spite of this, they have never (to my knowledge) recommended that you block all mobile content from desktop bots—only that you redirect mobile users properly if you choose to redirect them and that you block carrier-specific mobile content, which they do consider duplicate content.

This last recommendation has since been removed from Google’s mobile webmaster guidelines, so I can’t link out to it, unfortunately. Maybe someday Google and other engines will explain this in a way that is in accordance with their search results, but for now the way to get most visibility to your mobile content is to let it be returned by both mobile and desktop search.

If you really want to be optimized for mobile search, get out of the mindset that mobile content should even be considered duplicate content. Granted, most sites are going to have smaller versions of their desktop content on their mobile sites, but if that’s all you have you’re not taking advantage of the unique properties of the mobile user experience.

At SMX West this past March, Microsoft CEO Steve Ballmer talked about a new class of mobile-specific queries, and Google Analytics guru Avinash Kaushik talked about serving a mobile search experience with the Nexus One that most marketers haven’t even begun to consider. Chances are, if you’re only presenting your desktop content in a new format and excluding it from the web search engines, you’re not considering the mobile user either, and probably aren’t optimized for mobile search.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: SEO | Features: Analysis | SEO: Duplicate Content | SEO: Mobile Search


About The Author: is the SEO Director at Vivid Seats, is an SEO veteran with more than 14 years experience both agency and in-house, and is a thought leader in permission marketing as a columnist and a frequent speaker on SEO and mobile marketing.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Michael Martin PMP

    I agree that the SEs won’t ding mobile versions as dup content especially as those versions would have minimalized code vs the desktop version & should have some mobile backend code identification.

    To be safe, I covered the basics of mobile coding implementation at Search Engine Journal last week –

  • ho_logos

    It’s not clear that we should breathe any easier even if we know that the engines are not counting mobile content as dup.

    For some long tail terms, I see both and appear on page one when doing a desktop search. This suggests two potential scenarios. First, that my m pages are competing with my www pages for serp placement for desktop queries. Second, that my www pages area competing with competitors’ m pages for serp placement for desktop queries.

    So, the dilemma is as follows:
    Either search engines treat m content as duplicate or not duplicate.
    If duplicate, then we run the risk of our m page content diluting the SEO weight of our www page content in the serps for standard searches.
    If not duplicate, then we run the risk of our m pages competing with our www pages in the serps for standard searches AND our www pages have to battle with competitor m pages in the serps for standard searches.

  • Cindy Krum

    Hey Bryson,

    Nice article!

    In some cases, I think it is a valid strategy, but I generally recommend a mobile robots solution only as recourse to indexing problems or when mobile content begins to out-rank traditional content in traditional search. I always tell people that it is a last resort, and to approach it with extreme caution. I don’t think that mobile content ‘should’ be considered duplicate content, but I acknowledge that it could still be counted that way if something goes wrong.


Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide