Last week, the Bing Webmaster blog published a post about how Bing handles rel=”next” and rel=”prev” attributes. On the surface, it seemed as though Bing was announcing that it now supported these tags in the same way Google does. Last September, Google announced support of the rel=”prev” and rel=”next” attributes to designate paginated content, which enables site owners to cluster multiple pages of content into single entities so that indexing and other values can be consolidated.

However, the conclusion of the post included the following, which didn’t align with Google’s treatment of these tags:

“Implementing these rel=”next” and rel=”prev” link elements doesn’t trigger a new visual treatment for your pages on our search result pages. It does, however, allow us to more comprehensively understand and index your content”.

I talked to Bing’s Duane Forrester to clarify how Bing treats these tags. He confirmed that Bing does not use these tags to consolidate pages of a series into a single entity. Rather, Bing may use this markup in two ways:to aid discovery of pages and to enhance the search results display (possibly with links to the next and previous page, for instance). Although it wasn’t clear if either was happening yet. He said:

“We may use our newly gained knowledge on your site’s structure to provide easy access to other sections of the paginated or sequenced content from our results pages in the future. In addition, webmasters implementing these elements may benefit from more comprehensive indexing over time as we apply our newly gained knowledge of a site’s structure to our indexing heuristics.”

So what’s a site owner with paginated content to do? The best bet is to implement the tags for Google’s benefit and as a bonus, get potential additional crawling or display improvements from Bing. If pagination of your content is a significant problem in Bing (which may be the case particularly for some large e-commerce sites), consider blocking excessive pages with a Bing-specific robots.txt directive or noindex meta tag.

I’ve written and presented often on pagination for SEO. Why so much attention on this subject?

The reason is simple: it can be a big, hairy deal for sites. It’s right up there with faceted navigation as one of the most problematic crawling and indexing issues for large-scale SEO. It’s a tactic (actually a set of tactics) that our teams are continually evolving, testing, and refining.

So it was “double prizes” when Google announced the HTML 5 element rel next/prev for pagination.

3 Overall Tactics For SEO Pagination

There are three primary tactics that we use for SEO pagination:

• Classic Method (using noindex)
• View All Method
• Rel Prev/Next Method

Each of these is detailed below.

Classic Pagination for SEO: Using noindex

I’ve already detailed this technique in full, so I’ll skip the nitty gritty. The important thing to realize is that using this method does not directly transfer any equity from a series of component pages to the primary, canonical page. Rather, as component pages get crawled and link back to the canonical page, that equity is (hopefully) transferred as a second-order effect.

We would generally not recommend using this method for pagination today, except for fringe cases. It’s perfectly fine and will not hurt a site; on the contrary, it will greatly help a site that has SEO pagination problems. But, there are now even better methods as we’ll discover.

The classic SEO pagination method uses noindex but does not directly consolidate equity.

View All Method

The most elegant method is to utilize a View All page. In this approach, all component pages rel canonical back to the View All.

There are a few requirements for this approach:

• The View All must load quickly; at least 3 seconds end-to-end. Maile Ohye pointed out at SMX West that even if load times are excessive, if the page can load progressively the user experience will not suffer as much (since content will be viewable on the page immediately).

At SMX West, a few folks complained when I mentioned 3.5 seconds as the maximum load time tolerable for View All pages. The truth is, this is a “real world” goal and while not ideal, reflects the actual load times that we see on large sites.

Just take a look at these ‘last mile’ load times on US retail sites to get an idea of what latency looks like out there. It’s not particularly pretty, but more than anything demonstrates the opportunity these sites have.

• 

  Site latency reports from Google Webmaster Tools

  Our analysis of 20 top ecommerce clients showed an average load time of just over 4 seconds. The fastest site was averaging 2 second load times, an exceptional result in this set. But it was more common to see load times above 3 seconds and well into the 4 second range. While the average load time was 4.2 seconds, the slowest site loaded in over 9 seconds!

Another requirement for the View All method is to ensure all products, or items, that are included on the component pages are featured on the View All itself.

This ensures that there won’t be anything left out of the crawl, as pages annotated with rel canonical tags will not necessarily have links within their HTML crawled. It will also ensure there is a relevant match between what is being folded together in the paginated series.

View All method elegantly passes link equity to the canonical

The benefits of this approach are two-fold:

1. Users tend to love view all pages. In our experience and testing, pages with a lot of products or items all featured at once convert much higher than landing pages with a smaller selection of products. But the pages need to be fast.
2. All component pages in the series transfer their equity to the View All in a fairly direct fashion.

Also something to be aware of: Google will attempt to use your View All page by default, all things considered, when there are no other proactive signals in place. Be aware of this and take steps to control the SEO experience proactively yourself.

The Rel Next/Prev Method

The most current technique for SEO pagination makes use of the HTML 4/5 link element rel=”next” and rel=”prev”. The specifics of this implementation are well detailed in this Google support page, so let’s focus on the benefits and results.

It’s been our experience (especially with e-commerce clients) that it can be difficult to get a View All implemented as the canonical and default page. Merchandising teams don’t always like them; they don’t make holiday or seasonal specials as easy to manage; advanced landing pages can be better looking and UX and content teams often prefer them; they can make spotlighting certain products more difficult; and many other reasons.

Because of these challenges, rel next/prev is often an excellent method for handling pagination.

The benefits of this approach are as follows:

1. All component pages share their equity with the series. What does this mean? Basically, when page 9 of a series gets a link with rich anchor text, that equity is shared across the series with all the other pages. That’s a good thing.
2. However, using rel next/prev doesn’t prevent a component page from displaying in search results. So while these pages will “roll up” to the canonical (or default) page 1, they could still fire at search time if the query was relevant for that specific page. At SMX West, Maile assured us that it would be a very rare thing for that situation to occur. But it could occur.
3. Because of this, an additional recommendation (strictly as an optional step) is to add a robots noindex, follow to the rel prev/next component pages. This would ensure that component pages would never fire at search time.
4. Finally, all rel next/prev pages should also have a self-referencing rel canonical tag. In cases where tracking IDs are appended to a URL, these rel canonical tags will ensure no duplication and equity leak occurs.

Ensure implementation of rel next/prev is thorough

Conclusion & Pagination Recommendations

SEO pagination needs to be recommended situationally (like so much of SEO). Here are my recommendations:

1. If you have a fast loading View All page, and that page contains all the products and/or items included across the component pages, use this method. All component pages rel canonical to the View All, and it becomes your default ranking page in SERPs. It’s elegant, simple, and efficient. It will also best pass equity from each page to a single, canonical URL.




2. If you don’t have a quality View All, or your company doesn’t want to use that as the canonical URL, implement the rel next/prev methodology instead. This method will consolidate signals across the series, rather than concentrate them on a single URL; however, the end result should be the same, if implemented well: the canonical, ranking URL (normally page 1) will be given the equity. There is a substantial benefit in using this method over the classic noindex approach: equity is actually transferred to the series itself.
   
   Remember, the classic method does not directly pass any equity – there are no signals to do so – rather it achieves the same ends by opening up the crawl of component pages and keeping them out of the index and from competing with the ranking URL. Be aware that with rel next/prev, component pages can still fire at search time (although unlikely). You can optionally use a noindex, follow as well to avoid this. Ensure all pages have self-referencing rel canonical tags.




3. There are edge cases where the classic noindex method of SEO pagination is still viable. These are in situations, for example, where it’s important to address Bing consistently along with Google (Bing does not yet support rel next/prev), or when HTML 4/5 elements are not yet ready to be deployed at an organization. In cases like these, the classic noindex method is still a good option.

No doubt this will change again, but here’s the latest for your SEO campaigns. Best of luck and please let me know in the comments your experiences and insights.

Updates: Google’s Maile Ohye has recently published a video on pagination and SEO. Be sure to check it out. Vanessa Fox also covers the details in her thorough treatment of the topic, Implementing Pagination Attributes Correctly for Google.

How the Pagination Attributes Work

The pagination attributes can be used for any set of content that spans multiple pages. Typical scenarios include multi-page articles, product listings, and forum discussions.  Simply use the rel=next and rel=prev attributes to link all pages in a series together. For the following set of pages:

• www.site.com/products?page=1
• www.site.com/products?page=2
• www.site.com/products?page=3

The pagination attributes would be as follows:

Page 1:

<link rel=”next” href=”http://www.site.com/products?page=2″>

Page 2:

<link rel=”prev” href=”http://www.site.com/products?page=1″>

<link rel=”next” href=”http://www.site.com/products?page=3″>

Page 3:

<link rel=”prev” href=”http:// site.com/products?page=2″>

When To Use Pagination Attributes Instead of Canonical Attributes

Some sites are set up to use the canonical attributes to point all pages in a series to page one. As Maile points out in the video, this isn’t the correct use of the canonicalization tag (in part because Google only indexes the content on the canonical page, so any content from the rest of the pages in the series would be ignored).

If the paginated content is a subset of the canonical page (such as when you have a view all version or a filtered result set) or is identical (such as when the sort order changes the display but not the content), then use the canonical attribute instead of the pagination attributes.

General Best Practices

Use Absolute URLs

The href values can be absolute or relative (the original version of this article said they had to be absolute, but that version was incorrect). But using absolute URLs is a best practice, both to combat scrapers and in case URLs are duplicated accidentally across directories or subdomains.

The Chain Can’t Be Broken

The rel=”next” and rel=”prev” values must match. If they don’t, the chain is broken. For instance, for the following pages:

• www.site.com/products?page=1
• www.site.com/products?page=2

The rel=”next” attribute for page=1 must point to page=2 and the rel=”prev” attribute for page=2 must point to page=1.

The pagination attributes can only link together URLs with matching parameters. For instance, the following URLs aren’t considered part of the same series, as the second URL would break the chain:

• www.site.com/products?page=1
• www.site.com/products?page=2&referrer=twitter
• www.site.com/products?page=3

This means that ideally, you should dynamically insert the pagination values based on the fetched URL. In the case of the above example,when Googlebot fetches the page as:

www.site.com/products?page=2&referrer=twitter

The pagination values should be (dynamically inserted as):

<link rel=”prev” href=”http:// www.site.com/products?page=1&referrer=twitter”>

<link rel=”next” href=”http://www.site.com/products?page=3&referrer=twitter”>

Each page can be in only one pagination chain

• A page can’t contain multiple rel=”next” attributes.
• Multiple pages can’t have the same rel=”prev”.
• A page that contains a rel=”canonical” attribute to another page can’t be part of the canonical URLs paginated series. It must be paginated with the URLs that match it (and then Google will use the canonical attributes to consolidate each page accordingly). See more on how this works at the end of this article.

Advanced Techniques

Product listings in particular often have additional complexity, such as sort orders and filtered navigation. It’s best to start with the simplest paginated series and then make canonicalization and pagination decisions for each level of complexity. You can’t specify one view/filter as canonical and point all other versions of the paginated series to that default set.

Viewing and Sorting Options

If a set of product listings has multiple view options and those listings span multiple pages, you have to create a pagination set for each view option separately (since the pages aren’t subsets of a default version). For instance, if you provide options to view 20 products at a time or 100 products at a time and to sort by newest, price, and ratings, you would need to implement a separate paginated series for each view option. As you might imagine, this could cause you to end up with a large number of paginated series’. Shown below are just two of the many possible:

Filtering (AKA Faceted Navigation)

Things get even more complex when you introduce filters.

• If the filtered view is a subset of a single non-filtered page (perhaps the view=100 option), you can use the canonical attribute to point the filtered page to the non-filtered one. However, if the filtered view results in paginated content, this may not be viable (as each page may not be a subset of what you would like to point to as canonical).
• If you want the filtered view to rank separately from the default view, you would create a paginated series for each filtered category. You would need to also paginate all of the various sort and view options separately. Take REI.com as an example. The site has a Snowboards section that would likely be paginated and (hopefully) rank for [snowboards] queries. But a filtered view of just Women’s Snowboards, while a subset of the snowboards content would likely be paginated separately in order to rank for [women's snowboards] queries. (On the other hand, REI probably doesn’t want the filter by size version to rank separately, so that variation could be canonicalized.)

Using the Canonical Attribute in Conjunction with the Pagination Attributes

Each URL that’s paginated separately should also contain a canonical attribute. In the case of the earlier example with the optional referrer parameter, for instance, Google will first consolidate the default paginated series separately, and then the paginated series that contained the referrer=twitter parameter separately, but then use the canonical attribute of the pages to further consolidate the pages to the default version. That means that the URL of:

www.site.com/products?page=2&referrer=twitter

Would end up with the following markup:

<link rel=”canonical” href=”http://www. site.com/products?page=2″>

<link rel=”prev” href=”http://www. site.com/products?page=1&referrer=twitter”>

<link rel=”next” href=”http:// www.site.com/products?page=3&referrer=twitter”>

Too Confusing To Implement?

I admit, all of this sounds confusing. But it’s not so bad if you take things step by step:

1. What is the canonical version of each URL? Add the canonical attribute to the pages.
2. What is the default view for each paginated series? Add the pagination attributes to these pages.
3. What views/filters are subsets of broader views? Add the canonical attribute to these pages to point to those broader views.
4. What views/filters are not subsets of broader views or you want to rank separately? Add separate pagination attributes to these pages to make each a separate series.

You may be thinking about joining the website blackout movement, but yikes … what about the SEO implications? How do you take your site offline in protest without messing up your visibility in Google’s search results?

Well, Google’s Pierre Far shared several tips earlier today on Google+ in a post called “Website outages and blackouts the right way.”

In short, the advice is to use a 503 HTTP status code to tell spiders that the website is temporary unavilable. With a 503 status, Google won’t index the content (or lack thereof if you’re blacking out your site) and it won’t consider the site as having duplicate content issues (when all of the pages are blacked out).

But Far adds a couple important caveats to this advice regarding the robots.txt file and what will happen in Webmaster Tools if Google finds your site blacked out. Another Googler, John Mueller, adds additional information in the comments, so you’ll want to read the original Google+ post if you’re thinking about blacking out your website this week for SOPA, or in the future for any other reason.

Of course, also keep in mind that Bing may not handle things the same way if you do blackout your site.

New Monthly Search Update

The news comes from a post to the Google Inside Search blog, itemizing ten search-related changes that have been made.

Google did a similar post like this last month, and now it confirms this will be a monthly update on what it considers to be noteworthy changes but ones not big enough to merit blog posts of their own.

From the post:

Today we’re publishing another list of search improvements, beginning a monthly series where we’ll be sharing even more details about the algorithm and feature enhancements we make on a near-daily basis…

We’ve been wracking our brains trying to think about how to make search even more transparent. The good news is that we make roughly 500 improvements in a given year, so there’s always more to share. With this blog series, we’ll be highlighting many of the subtler algorithmic and visible feature changes we make. These are changes that aren’t necessarily big enough to warrant entire blog posts on their own.

While Google calls these all algorithm changes, some of them are really related to the search interface, while others impact how Google crawls, which is different from the algorithm that controls how Google ranks pages (to understand more about search algorithms, see our What Is SEO / Search Engine Optimization? page and watch the movie).

On to the changes:

Parked Domains Get Ticketed

One of the most significant changes is that Google says it has a new algorithm to detect parked domains. From the post:

New “parked domain” classifier: This is a new algorithm for automatically detecting parked domains. Parked domains are placeholder sites that are seldom useful and often filled with ads. They typically don’t have valuable content for our users, so in most cases we prefer not to show them.

That’s a pretty easy change to understand. Many domainers I’ve spoken to have understood over the years that it’s become harder to rank on Google without having some substantial content on their sites. This is a clear sign life is getting harder.

Ironically, Google’s own AdSense For Domains program has fueled much of the parked domain industry that its web search team is now penalizing against.

Rewarding The Original

Another big change is that Google says it can now better detect which is the “original” page when confronted with several that seem similar:

Original content: We added new signals to help us make better predictions about which of two similar web pages is the original one.

Within a web site, a variety of things can cause a page to be duplicated. However, a bigger issue for many publishers is when people copy or “scrape” their content without permission. These scraper sites sometimes can even outrank the original site for searches.

Google doesn’t specifically say this change is aimed at scraper sites, but it should help with that issue — and it’s an issue Google’s been especially been battling against since launching its Panda Update earlier this year.

Our previous post from August also talks more about this battle: Google Signals Upcoming Algorithm Change, Asks For Help With Scraper Sites.

For Google, it’s also another reason why publishers may want to consider using the canonical tag. The posts below have more about this:

• Google, Yahoo & Microsoft Unite On “Canonical Tag” To Reduce Duplicate Content Clutter
• Google Supports Cross-Domain ‘Canonical Tag’
• Do You Have Duplicate Content Issues Across Domain? Google Will Now Alert You

Stop Crowding Me

A third big change was Google saying that it’s pulling back on allowing a single site to potentially occupy too much of the top search results. From the post:

Top result selection code rewrite: This code handles extra processing on the top set of results. For example, it ensures that we don’t show too many results from one site (“host crowding”). We rewrote the code to make it easier to understand, simpler to maintain and more flexible for future extensions.

We’ll try for a follow-up here to get further clarity, but about a year ago, Google made it possible for one site to have more than the usual single listing it might get at the top of the page. These stories, especially the second, explain more about this:

• Official: Google Now Lets One Domain Dominate Search Results
• Google Showing More Results Per Domain For More Queries

The change means that that a brand owner may occupy less of the search results page for a search on their name, so competitors or critics potentially turn up more.

Of course, brands like McDonald’s or Coca-Cola have so many additional sites, along with social media profiles, that they still do well in crowding out others.

My post from September, Should Rick Santorum’s “Google Problem” Be Fixed?, explains this more, at the end.

Rare Words Count For More

An interesting change is that if you’re searching for a “rare” or unusual word, Google is easing back on ignoring this and potentially returning matching web pages that might not contain that word.

From the post:

Sometimes we fetch results for queries that are similar to the actual search you type. This change makes it less likely that these results will rank highly if the original query had a rare word that was dropped in the alternate query. For example, if you are searching for [rare red widgets], you might not be as interested in a page that only mentions “red widgets.”

Bigger & Fresher

Elsewhere in the post, Google says that it is doing “more comprehensive indexing,” promising that this will make “more long-tail documents available in our index, so they are more likely to rank for relevant queries.”

Google also said that its blog search results are both more comprehensive and fresh. Image results were also said to be fresher.

Suggestions, Tablet Layout & Goal!

In the remaining changes, first, Google says that it will be offering more autocomplete suggestions. Second, it says it has made small changes to improve its look on tablets.

Finally, those looking for Major League Soccer and Canadian Football League scores, rejoice! Google says it will now display scores, schedules and links to game recaps and box scores for games.

Related Stories

• What Is SEO / Search Engine Optimization?
• Periodic Table Of SEO Ranking Factors
•  Google Signals Upcoming Algorithm Change, Asks For Help With Scraper Sites
• Do You Have Duplicate Content Issues Across Domain? Google Will Now Alert You
• Google Showing More Results Per Domain For More Queries
• Responding To Complaints, Google Adds Verbatim Search Results
• Improved Snippets, Rank Boost For “Official” Pages Among 10 New Google Algorithm Changes

“When we discover a group of pages with duplicate content, Google uses algorithms to select one representative URL for that content. A group of pages may contain URLs from the same site or from different sites.”

They note that when they choose a representative URL from a different domain, they call this “cross-domain URL selection”.

In cases where multiple URLs contain the same content (for instance, due to infrastructure configuration, optional parameters, syndication, or internationalization), many options exist for site owners to indicate to Google which version is canonical.

However, in some cases, the site owner doesn’t use these options to specify a preferred version or Google may select a different version than the site owner specifies.

This new feature alerts site owners  when their “algorithms select an external URL instead of one from their website”. They say common reasons for this include:

• Site owner-specified – if you’ve moved your domain or have implemented the rel=canonical attribute to indicate that a page on another domain is canonical, then this alert is simply confirmation that Google is indexing as you’ve specified.
• Regional sites – if you have the same content on multiple regional sites (for instance, the same English content on a .com (for US), a .co.uk, and a .com.au), Google may cluster pages with identical content across sites and use relevance signals to determine which to display per query.
• Incorrect canonicalization – in this case, a page may inadvertently use the rel=canonical attribute to specify a page on a different domain as canonical.
• Misconfigured server – a hosting misconfiguration (this in particular happens sometimes with shared hosting) may cause a two different domains to display the same content)
• Hacked site – sites are sometimes hacked to point to other domains.
• Scraped content – the blog says that “in rare situations”, Google may select a URL from a site that has scraped your content.

Duplicate content is a big problem for SEO for a number of reasons. If you’ve been following this multinational series, you’ll know that some of the biggest gains to be had in SEO today come from fine tuning your SERP sales message to make the landing page you target to a search term convert at a much higher rate than average.

If you are unable to ensure that your landing page is the only page optimised for that term, then you lose that strategic ability.

In short: regardless of the SEO benefit to be had by eliminating duplicate content, the impact of losing the ability to control which page turns up for which search term is a greater concern.

Automated Translation & Duplication

This is a common outcome, especially when looking at multinational websites where the translation in some countries is direct from another language, often as the result of an automated translation.

Matt Cutts has gone on record to state that such translation would be considered duplicate, and so many webmasters are getting into difficulties despite trying to do the whitehat thing and generate relevant content for the country.

Auditing For Duplicate Content With Google

Of course, Google gives you some idea of what other content it considers duplicate via its Webmaster Tools, with The HTML suggestion report highlighting areas of duplication in title tags and meta descriptions.

But this is really only useful as a refining tool and won’t indicate when, for example, your e-commerce system generates new pages for every available colour for every item of clothing you sell; if the colour is included in the title and meta description (generating 10 pages for domain authority to be spread across – therefore struggling –  to compete for a competitive product when one single page would perform significantly better).

Similarly, it won’t tell you if your site used frames to host the bulk of page content, excluded from spider access via the robots.txt while the frameset pages are correctly tagged for SEO.

And so on. But most importantly, it won’t help you dig into a site to identify ‘crawler traps’ created by inadvertently creating navigation loops the generate near infinite permutations of URLs to be indexed, causing search engines to simply abandon crawling at a certain point – as defined by its perceived authority of your domain.

That means that if your site is considered low authority, or if you are hosting a large number of pages on a medium authority site, then page parsimony is critical to get the best possible performance out of your landing pages.

So what’s the best way to quickly narrow down duplicate content issues you may have?

Simple: unleash the power of chained Google site:, inurl: and  intitle: operators.

If you haven’t read about search operators, this is a good place to start, but there are a few that aren’t listed in there which we can look at in more depth in a future article. For now, lets see how we hunt down duplication.

Finding Site Architecture Duplication

Lets take a look at Hilton Hotels’ global site architecture as an example audit.

First up, run a quick site: command, setting results to 100 so you can easily browse through the results: http://www.google.com/search?num=100&q=site%3Ahilton.com.

Running that search today, I get 388,000 results.

If I click through to the last paginated page Google offers me, we can also see that after 848 results, Google has truncated the results suggesting that the rest are ‘very similar’ to the listings it has already displayed; i.e. the site contains massive amounts of duplicate content.

If I were to click on the link above, then Google would append &filter=0 to my search URL.

Those reading who’ve been around search for a long time will recognise that as the old ‘Supplemental listing’ parameter discussed by Matt Cutts and largely ignored for the last few years since the tag highlighting supplemental results was dropped.

We won’t go into supplementals further in this article, but it’s important to note the supplemental index was created to handle the mass duplication frequently encountered by Google in attempting to index the whole Internet.

The important attribute of supplemental listings to be aware of is that it can’t outrank any other page in the main index returning for a competitive search query.

So, hilton.com seems to have a pretty severe problem. Can we work out what’s happening, just by querying Google?

Well, to a large extent, yes. We can.

Just quickly browsing down the first page of results, I can see frequent repetition of Hilton Doubletree Results as non-www hilton.com results. So, for example, the following are both identical:

• http://hilton.com/en/dt/promotions/dt_greenhotels/index.jhtml
• http://doubletree.hilton.com/en/dt/promotions/dt_greenhotels/index.jhtml

Also, there’s a heck of a lot of jsessionid parameters, and sure enough we can chain our search operators to get an idea of the full extent of that issue:

• http://www.google.com/search?num=100&q=site%3Ahilton.com+inurl%3A%22jsessionid%22.

Ouch! By chaining the operators, we in fact get a larger number of indexed pages that Google would originally admit: 587,000.

For hilton.com, we can start to identify other duplication issues by making excluding the jsessionid issue from our site operator search simply by appending a minus sign ‘-’ to our search chain.

Using this filtering technique, we can carry on using search operators to find that:

• There’s massive duplication from the secure site (https) being indexed while selecting hotel availability dates:
  • http://www.google.com/search?num=100&q=site%3Asecure.hilton.com+inurl%3Achoose_dates%22
• Their special offers are indexed and are poorly differentiated:
  • http://www.google.com/search?num=100&q=site%3Ahilton.com+intitle%3A%22Special+Offers%22

And so on… leaving us with a search chain filtering out those issue so we can keep combing the results for more problems.

• http://www.google.com/search?num=100&q=site%3Ahilton.com+-inurl%3A%22jsessionid%22+-inurl%3A%22choose_dates%22+-intitle%3A%22Special+Offers%22

There’s no silver bullet solution, although canonical tags would go a long way to solving some of the more severe issues. But a concerted effort to solve these duplicate content issues with a well documented business case for the various change requests required would return huge improvements in Hilton Group’s Web presence and deliver immediate, significant bottom line impact.

Using Google’s view of your domain to identify issues with your domains and take back control of your landing pages to better convert your search engine traffic to sale is a solid approach, in any language.

When I took a random sample of 11 of the most popular mobile queries and examined the characteristics of the top 5 Google smartphone results for each query, the average domain authority (using SEOMoz metrics) for the first five positions was 83/100, and the first result (on average) had as many links pointing to the first listing as it did to listings 2 through 5.

Furthermore, there was only one site in the sample that ranked with fewer than 10 links, and three quarters of the total sample had more than 1,000 total links. As with desktop search, it appears linking and link building matters in mobile search.

Historically, conversations around mobile link building have been sparse, as mobile sites have been seen more as a link equity problem for brands, rather than as something that can help them gain links.

Performics, for example, recently compared mobile sites to print URLs, in that they both contribute to reduced link equity as a result of duplicate content. John Mueller of Google also recently recommended using the canonical tag on mobile sites that duplicate desktop content.

These recommendations are fine if all you’re doing with your mobile site is duplicating desktop content. However, brands that simply use their mobile site to reformat desktop content for mobile users are missing the point.

If you’re old enough to remember pre-bubble early Internet days, when some brands thought it was okay to put PDFs of your print brochures online and expect them to do as well in a digital context, you should know better.

Site owners want to link to content that is unique, innovative and unusual, and cutting-edge mobile sites can be just that. Break out of the mobile sites as duplicate content trap and follow these guidelines to possibly get more links to your mobile site than your desktop site.

1.  Don’t Put All Your Eggs In The App Basket

If you’re going mobile, make sure your mobile strategy includes a mobile site, and not just apps. Apart from the many problems inherent with focusing on mobile apps as a way of gaining a mobile audience, mobile sites appear in search results for more queries, and link equity to the sites will ultimately benefit your domain, and not just the app stores you submit to.

2. Publicize With Mobile Directories & Awards Sites

Just having a mobile website is enough to get you additional links from certain sites that focus on mobile content. If you have a really good one, letting sites that cover mobile content know about it could get you listed in a top mobile sites story, such as this one that ran on Mashable last year.

If it’s exceptional, you could get an additional link from Time magazine or the Webby Awards. And if it’s maybe not exceptional but not outright embarrassing, you can get a link from WAP Review, Mobile Awesomeness, or one of the many mobile directories that have emerged with the mobile Web.

It’s worth noting that all of these links are only available to brands that have built mobile content, so if you haven’t yet, it’s not yet too late to start.

3.  Use Mobile Information Architecture & Content

I recently looked at Staples’ mobile site, which is one of many sites that mobilizes its desktop content using a transcoder, rather than builds content specifically for mobile users. Because searchers often search using the same keywords, but with different frequencies, this is often a missed opportunity to address the mobile user experience of the mobile site.

For example, with Staples’ mobile site, it’s clear from the branded mobile queries found in the Google Keyword Tool that when it comes to Staples, searchers overwhelmingly care about finding information about their local Staples store.

Figure 1 Mobile queries of searchers looking for Staples brand filtered by mobile intent and sorted by volume.

Smart marketers would use this information to their advantage, and give the searcher not just a stripped down version of their desktop site, but also a mobile experience that highlights the locations they’re looking for and uses the unique properties of mobile to make a connection to the stores as soon as possible.

For example, instead of including a separate page adjacent to the home page with a store locator that requires the user to input his/her information, the home page could easily use the unique properties of mobile to find the user’s location based on GPS and display a map and contact information for the nearest Staples to the user.

It could also link to an optimized store page that would give local shoppers a list of products on sale at the local store with the option to buy and pick up in store. Perhaps Staples could even include mobile coupons to redeem in-store or time-sensitive deals that could be redeemed on the device at the nearest Staples.

These things would be of value to mobile users, and they would help brands get links to content that they can’t get just by transcoding their desktop site for mobile users.

4.  Use Mobile Functionality Desktop Users Would Envy

I saw Alan Moore of SMLXL speak at a mobile conference a few years ago, and he told a story about a site in Japan that allowed mobile users to map the shortest path to their destination in a rainstorm that passed under as many awnings as possible.

There are apps now like Shazam that use the strengths of mobile phone and design around them, rather than taking something that was built for the desktop and making it mobile. Google recently announced that their mobile visual search technology is available in  desktop search, but it started as a mobile innovation that made desktop computer users want to try.

If one of the great truisms of SEO is “content is king”, why wouldn’t that apply to mobile content as well?

The best sites play to the strengths of mobile in their design, and compel users to share and link to them. What is it that you can provide your users that would be so useful to them while mobile that they are compelled to share, link to and write about it?

If you can’t answer that, it may be time to go back to the drawing board and provide something useful that doesn’t yet exist on your desktop site. Most modern browsers now have access to the gyroscope, camera and accelerometer. Use them to create something unique that mobile users will find invaluable, and the links will find you.

5.  Don’t Put Canonical Tags On Your Entire Mobile Site

Yes, if you duplicate desktop content in a mobile format, the best practice today is to use CSS to reformat the content on a single URL or use canonical tags to redirect the mobile link equity.

However, if you just do that you’re missing out on a lot of mobile traffic and links, as I hope I’ve demonstrated in this article. Better to add canonical tags to transcoded desktop content, and build unique content for mobile users that will help you be found when they search.

My original article on parameter handling describes the issues that parameters can cause with how search engines crawl and index a site. In particular, as the parameters multiply, the number of near duplicate pages grows exponentially and links may be coming in to all of the various versions. This dilutes PageRank potential, and the “canonical” version of a page may not ending ranking as well as it otherwise would. Search engines may also spend much of their crawl time of various versions of a small subset of pages, preventing them from fully crawling (and thus indexing) the site.

New Options for Providing Google Information About the Parameters In Your URLs

So what’s new? You can now specify whether or not a parameter changes the content on the page. For those that don’t, once you specify that, you’re done! Things get more complicated if a parameter changes the content on the page. You now have a number of options available, described in more detail below. This latest incarnation of the feature impacts both which URLs are crawled and how the parameters are handled.

To access the feature, log into your Google webmaster tools account, click on the site you want to configure, and then choose Site configuration > URL parameters. You’ll see a list of parameters Google has found on the site, along with the number of URLs Google is “monitoring” that contain this parameter.

The default behavior is “Let Googlebot decide”. This results in Google figuring out duplicates and clustering them. As they say in their help content:

“When Google detects duplicate content, such as variations caused by URL parameters, we group the duplicate URLs into one cluster and select what we think is the “best” URL to represent the cluster in search results. We then consolidate properties of the URLs in the cluster, such as link popularity, to the representative URL. Consolidating properties from duplicates into one representative URL often provides users with more accurate search results.

To improve this process, we recommend using the parameter handling tool to give Google information about how to handle URLs containing specific parameters. We’ll do our best to take this information into account; however, there may be cases when the provided suggestions may do more harm than good for a site.”

But if you want to have some control over that process, click Edit to specify whether that parameter changes the page content. You can look at a list of recently crawled URLs with that parameter to help you figure out what they’re used for.

Your choices are:

• No: Doesn’t affect page content
• Yes: Changes, reorders, or narrows page content

If a Parameters Doesn’t Page Content

If you choose no, you see the following message:

Google will associate any URLs with those parameters with the versions without them. So for instance, if Google comes across the following two URLs, they’ll cluster both together as the first URL:

• www.mygreatsite.com/buffy-is-still-awesome.php
• www.mygreatsite.com/buffy-is-still-great.php?referrer=buffyangelship4eva

If a Parameter Changes Page Content

If you choose Yes, you see even more options.

How does this parameter affect page content:

• Sorts – the content stays the same, but is reordered
• Narrows - the parameter is a filter that displays a subset of the content
• Specifies - the parameter determines the page content
• Translates - the page content appears in a different language
• Paginates - that parameter indicates the page number of a larger set of content
• Other - none of the above

Which URLs with this parameter should Googlebot crawl:

• Let Googlebot decide -this is the default; Google will use other signals to determine which URLs to crawl
• Every URL - Google should crawl all URLs and use the value information above to choose what to consolidate
• Only URLs with value – triggers a list of values Googlebot has crawled to  choose from; use this option if you want Google to crawl only URLs that contain a specific value for a parameter (and not crawl any others)
• No URLs – you don’t want Google to crawl any URLs with this parameter (but not that using a pattern match in robots.txt may be better option in this case)

Handily, Google shows you how your choices will impact crawling:

Seem overwhelming? Fortunately, another new feature Google has launched is the ability to download all of the parameter settings as a CSV file so you can sort though them offline.

Should You Use This Feature?

As Google notes on the page, “Only use this feature if you feel confident about how parameters work for your site. Telling Googlebot to exclude URLs with certain parameters could result in large numbers of your pages disappearing from our index.” The benefit, though, is multifold. The crawl is more efficient, so more pages of the site may get crawled and therefore indexed, and links are consolidated to a single version that may then gain rankings.

My guess is that Google may not immediately stop crawling the pages, as they may want to make sure the content is in fact the same before consolidating the URLs into a single version. But if you aren’t an expert with URL parameters and canonicalization, you probably want to let Google sort this out and not change these settings. They do, in fact, do a fairly good job, particularly when URLs use standard key-value pairs.

If you know the ins and outs of URL parameters and know exactly what you want Google to crawl, then go for it. Jonathon Colman of REI jumped on these new features right away!

Should you use this feature if you’re already using the rel=canonical attribute on the pages? Sure! Google’s help says:

“You can also give Google additional information by adding the rel="canonical" element to the HTML source of your preferred URL. Use which option works best for you; it’s fine to use both if you want to be very thorough.”

I still have questions about this feature, like how it impacts a crawlable link architecture with substantial pagination. If you’re interested in digging as well, note that I’ll be moderating a session on this at SMX East, and Maile Ohye of Google, who’s spent a lot of time on this issue, as well as experienced practioners, will be on hand to provide practical answers.

I’ve pulled out the key points from this post in bullet format:

• Google differentiates between traditional mobile phones and smartphones.
• Google has two bots: Googlebot and Googlebot-Mobile. Googlebot crawls desktop-browser type of webpages and content embedded in them and Googlebot-Mobile crawls mobile content.
• Currently only traditional phones are supported with special useragent strings within Googlebot-Mobile, not smartphones (this may change)
• Google said they “expect smartphones to handle desktop experience content so there is no real need for mobile-specific effort from webmasters.
• It does not mean you can’t serve a special style sheet to smartphones, Google said, “the decision to do so should be based on how you can best serve your users.”
• URL structure: For Googlebot and Googlebot-Mobile, it does not matter what the URL structure is as long as it returns exactly what a user sees too.
• Using the same URL “is not considered cloaking by Google.”
• Mobile sitemaps: you should include only mobile content URLs in Mobile Sitemaps, even if these URLs also return non-mobile content when accessed by a non-mobile User-agent.

This blog post was in response to a question I had in regards to a video Matt published, which is below. But make sure to understand that you do not need to use a mobile specific URL, you can serve up mobile content on the same URLs you serve up to desktop users. In addition, it is important to note that smartphones like iPhones and Android phones are not necessarily and currently considered “mobile” devices to Google or Googlebot-Mobile.

Related Stories:

• The New Mobile SEO: What You Need To Know
• Two New Mobile SEO Tips For 2011
• Has The iPhone Made Mobile SEO Obsolete?
• Mobile SEO Tip: Transcoding Services Can Dilute Link Popularity
• Don’t Penalize Yourself: Mobile Sites Are Not Duplicate Content
• Why Mobile Searchers Need Mobile-Optimized Sites
• Sorting Out The Mobile Search & SEO Mess
• Google Writes On Mobile Site SEO Concerns & Techniques