Pagination has always been a sticky problem for search engines. While not nearly as complex as faceted navigations for SEO, they can certainly cause crawling inefficiencies and excessive duplicate content. They can also create problems with undesirable pages ranking for important terms, in cases where the search engines don’t pick your preferred URL.

Fundamentally, pagination is an interface problem. A computer should have no problem processing hundreds (if not thousands) of items on a single URL. But people can’t read that much stuff, and long scrolling pages can be a poor user experience.

Here’s where user experience and search engines have a face off, and the user wins: enter pagination.

Historically, we have handled this problem several different ways. Like so much with SEO, the specific implementation depends on a number of factors. We typically use a combination of the following tools:

  • rel=”canonical” tags
  • meta robots noindex
  • parameter handling tools
  • robots.txt (rarely, it’s a sledgehammer)
  • making pages unique (title, meta description, URL)

Today we can add to this toolset:

  • rel=”next”, rel=”prev”

One area that has changed recently is our understanding (through testing, and messaging from Google) of how crawlers interact with pages tagged with rel canonical. Quoting from my article about Maximizing SEO Product Visibility,

“We have traditionally operated on the assumption that URLs annotated with rel canonical wouldn’t be fully crawled. In other words, links within pages wouldn’t be followed, anchor text and PageRank would not be passed, and the URL would simply be ‘soft 301′d’ to its canonical target. However, that may change based on the latest information from Matt Cutts. …

“Cutts recently said that links on pages annotated with rel canonical would still be crawled, based on the overall PageRank of the URL, among other factors. From this information, it appears rel canonical is a totally separate (i.e., distinct) process from crawling. …

“There is a fairly big ramification of this in how pagination is treated. Our methods for handling it typically employ ‘noindex, follow’ on paginated URLs (2, 3, 4, etc.), and no use of rel canonical except to self-reference in cases of duplicate URLs; certainly, no use of rel canonical to reference page 1, since that would prevent links on deeper pages to get crawled.”

No doubt some of that may now change with the introduction of rel next and prev. In fact, we’ve already begun testing use of rel canonical in concert with meta noindex.

Background On SEO & Pagination

Issues with pagination seem to occur most frequently on e-commerce sites, although the problem can pop up on publishing and information sites, too. Typically, we recommended companies default to their “View All” page for visitors. We’ve seen time and again that customers shopping online want to see all the inventory in front of them, at once.

This is why well-crafted search result pages perform so strongly and usually have a conversion rate that surpasses category landing pages with copy, images and more formalized layout. I was happy to see Google echo this advice recently when they announced their preference for View All pages in search results.

However, clients don’t always like this recommendation. First, it can cause latency, especially on large sites with hundreds (if not thousands) of products or items in a category. Second, merchandising and political interests are at play.

Having control over which products are featured on a category are important to these folks, and in some ways, the “View All” takes away this control, making it more of a level playing field for the products rendered. As always with SEO, the business needs take precedence, and recommendations must be catered to the specific situation.

Our Approach For SEO Pagination Problems

Now that rel=next and rel=prev are supported (which is awesome), our testing has begun. We don’t yet know how well this is working, but we will soon. In the meantime, we’re continuing to recommend the following strategy:

  1. Create a View All page (it isn’t necessary to make this the default view)
  2. Link to the View All from category- and product-level URLs. Messaging can be simple, something such as “view all products.”
  3. Here’s where we differ from Google: add a “meta noindex, follow” to the View All and all the pagination URLs. This effectively pulls them from the indices. (Note: We may revisit this strategy and modify it based on the success of rel next/prev and Google’s desire to feature View All URLs in their indices.) Additionally, add rel=”canonical” annotations to these URLs.
  4. Ensure paginated URLs are made unique: URL, page title and meta description. Why? Because this helps differentiate them and send quality signals. Google should then give more weight to the pages (and their links) if not only the content (e.g. the products listed) is unique, but also the structure of the pages.
  5. Add the View All and the paginated URLs to your XML sitemaps to ensure crawling. These can be removed after a period of time.

SEO pagination solution

Our SEO solution for pagination, category and product level.

SEO for pagination, product leve

Our SEO solution for pagination, product or item level.

This has worked very well for our clients. The result is that the default category URL (in essence, page 1) ends up ranking best for its targeted terms. Potential duplication is solved, and extra pages are removed from the indices, keeping things clean.

There are potential downsides, too. For one, pages annotated with meta noindex still need to be crawled, so it does nothing for crawl efficiency on very large sites. The second problem is that, with rel next/prev, this strategy (while successful so far) probably will need a refresh.

Finally, there is some concern that using meta noindex and rel=”canonica”l in tandem could cause issues, since the two signals are technically in conflict. Meta rel canonical says, “Soft 301 this to the canonical version and pass all scoring factors.” Meta noindex says, “Crawl this but don’t feature it in search results.”

However, judging by Cutts’ recent comments, those processes are actually separate and distinct. Therefore, they shouldn’t be in conflict. We’ll keep testing.

What are you seeing out there? I look forward to your comments and experiences.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: SEO | Enterprise SEO

Sponsored


About The Author: is the Chief Knowledge Officer at RKG, where he blogs regularly. You'll find him speaking at conferences around the world when he's not riding down mountains on something fast. Follow Adam on Twitter as @audette.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Anthony D. Nelson

    Great article Adam. I also recommend (when possible) having the client create more categories or subcategories, lessening the overall amount of products and pages in each category.

  • http://www.raisemyrank.com/ Bob Gladstein

    I’m not clear on why you’d choose to put a “noindex, follow” on the paginated pages and go to the trouble of giving them unique URLs, titles and meta descriptions. Are you suggesting that if they’re too similar they won’t even be crawled, keeping their links from being followed?

  • http://gplus.to/clarktaylor Clark Taylor

    Hmmm. Before the rel=next/prev announcement, I was going to implement a noindex,follow on all paginated pages. After hearing the discussion about this at SMX, I thought rel=prev/next was a better solution. Doesn’t rel=prev/next really accomplish the same thing…tells the SEs that paginated pages are part of a series? I think I remember the speaker from Google saying that they will most likely display the page with only a rel=next (the first page in a series) in the SERPS.

  • http://www.audettemedia.com Adam Audette

    Bob, great question. The thinking is that unique pages will be more likely to get scored favorably, rather than just counted as dupes and rolled up to a canonical selection (Google makes canonical decisions when pages appear duplicated). As such, the links on said pages should get crawled more frequently and/or be counted more heavily if the pages are made more unique.

    Quite possibly overkill, but you can’t be too cautious in my opinion, as long as the resources to do this are not excessive.

  • http://www.audettemedia.com Adam Audette

    Clark, I treated rel nex/pre in this article. Still testing, but yes, that likely changes things.

  • http://www.audettemedia.com Adam Audette

    Anthony – great point to increase the number of category types. Just need to a little careful with too much cross-categorization.

  • http://www.30go30.com/dr-pete Dr. Peter J. Meyers

    I have mixed feelings about this post. On the one hand, this a great, comprehensive overview. On the other hand, I was mid-way through an updated SEOmoz post about pagination that I’ve now decided to scrap. So, great post, but I hate you, Adam ;)

    Google’s mixed messages haven’t helped. They’ve been contradictory about using rel-canonical for pagination, and often what they say doesn’t seem to match how the algo actually works. Recently, they finally said that a canonical to “View All” was ok.

    One note. I asked some of the Bing crew about rel=prev and rel=next yesterday, and Duane Forrester said that they don’t support it yet. He implied they’re taking a wait-and-see approach.

  • http://www.seoinc.com/ Kevin Graves

    To clarify on step 3, when you say “add rel=”canonical” annotations to these URLs”, do you mean to self-reference the component pages itself for any possible duplicate content, or to reference the first page in the series?

    Should the view all pages be showing all of the ‘products’ from a particular ‘category’ rather than all products from all categories?

  • http://www.audettemedia.com Adam Audette

    Thanks, Dr Pete! Great compliment coming from you. I agree that Google’s mixed messaging has been a problem here, and also that some of their (now dated) advice didn’t seem to play out the way the intended out in the wild, at least in our experience.

    I wouldn’t expect Bing to adopt rel nex/pre anytime soon. They’re still not supporting rel canonical very well, from what we’ve seen, and have yet to support cross-domain canonicals at all. Keeping up with Google’s innovations is not an easy thing to do. But I like Bing’s approach to efficient crawls and important URLs rather than trying to index the entire web like Google.

  • http://www.audettemedia.com Adam Audette

    Kevin, in step 3 the rel canonicals are self-referencing, to avoid any duplication through permutations in the URL structures. Always a case by case call, but this solves problems with tracking parameters on URLs causing dupes, for example. Never add rel canonical targets back to the first page from deeper pages in the series.

  • http://www.seoinc.com/ Kevin Graves

    Hey Adam,

    I agree, and i’ve taken the same approach for the most part with my own personal websites. What separates your steps from mine is the introduction of the “view-all” page.

    Would you mind clarifying the purpose of having the “view-all” page in this set of steps for SEO? Why create a “view-all” page if the intent is not to have the page indexed or associate rankings with that page? I have read the google announcement about the preferred “view-all” page, and i agree in most cases it returns the most relavent result on there for users.

    Is there something besides simply having a “view-all” page that is contributing to your set steps for implementing the next/prev tags for users? I believe from Googles standpoint, if you dont have a “view-all” page is where you should then introduce the next/prev tags, otherwise those component or paginated pages should canonical to the “view-all” page.

  • http://www.audettemedia.com Adam Audette

    Kevin, the view all page simply creates a single crawl point for all the products/items/whatever listed in that category. The same products are listed on pages in a series, of course, but the view all is an additional step to ensure all the products get sufficient internal link weight and are crawled as equally as possible.

  • http://www.seoinc.com/ Kevin Graves

    Thanks for the reply,

    I agree mostly with your steps except creating the “view all” page if one doesnt exist.

    In the event that a “view all” page is created or exists, that is when the canonical tag should be used on the component pages to reference the “view all” page to consolidate the rankings to one page. Otherwise In the event that a “view all” page doesn’t exist, one doesn’t need to be created, and the webmaster should use the rel next and previous tags.

    In the cases that you want to ensure that all the products get sufficient internal weight and are crawled, then its okay to create that “view all” page, but then the canonical tag should just take precedence to reference that main page

  • Rahul Srivastav

    Great annotation on Pagination page, this would help to large website which is facing duplicate content, duplicate pages. But, it should be implemented properly, because paginated pages does matter in ranking. I liked this. Keep posting these kind of information.
    Thanks.

 

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide