Since Google’s Caffeine update to its server infrastructure back in June 2010, Google has stepped up the importance of page load speed to its core algorithm. But page load speed has always been important to Google in providing a good searcher experience for Google searches.

And this is a logical stance if you take a second to think it through.

After all, if you consistently found that despite throwing up relevant results your searches continually returned sites which took ages to load, you’d seriously consider using a different search engine too, right?

So what causes slow page load times?

Well, the top five most common causes (in no particular order) which are cited in most large-scale studies of page response times, and which are tackled by all of the tools out on the market today are:

• Poor Caching Controls
• High Request Overheads
• Large Payload Sizes
• Long Round-Trip Times
• No Device Optimisation

The details around each type of issue are well documented elsewhere, for example in Google’s Web Performance Best Practice or Yahoo! Developer’s performance notes, so I won’t run over them again here.

All we need to know is that the number one – and most common – impact to page load times is poor caching controls. However, for multinational sites the very specific caching issue we should look to solve is ‘Proxy Caching’.

Proxy Caching is the practice of storing static resources on public web proxy servers. This is the type of advantage we can leverage in multilingual campaigns targeting multiple territories using a well distributed CDN.

So what should we be looking at to evaluate the quality of a CDN?

1. Well Distributed Edge Nodes

The most important criteria. The location of the CDN’s Edge Nodes should be close to the searcher’s location, this means that having all their servers in just Europe or the US would disadvantage searchers from Asia, for example.

Track down the CDN’s network map and match it against the locations you’re targeting in your search marketing startegy: are all the key countries covered?

If not, you’ll likely still see multiple hops are required for page requests and much of the speed optimisation is lost.

Try testing a CDN’s claims by performing a Ping & Traceroute from an IP within one of your target markets. Use a free multi-territory ping service such as that offered by dotcom-monitor.

2. Configurable Static Caching Of Dynamic Content

You may have noticed that Proxy Caching is concerned with static content.

If your site is built dynamically (and if it’s targeting multiple countries and languages, then it almost certainly is), then regardless of how well your static assets are cached, if big chunks of your body HTML requires dynamic loading then speed gains are for naught.

So make sure your CDN allows you to configure only those essential dynamic areas that should be served directly from your own servers, and dump all the rest of the content to their caching controls.

In 99% of cases, ‘essential’ dynamic areas are often not all that essential, so take time to determine precisely what functionality you need to retain control of on your own servers, and allocate your own resources to focus on its delivery.

For example, updates on your homepage about latest news do not count as ‘essential’ dynamic content.

Depending on your flavour of code, you will likely find a function that allows you to specify CDN information for static files or cache content generated dynamically such as Django’s STATICFILES_STORAGE, which also gives you an easy way to switch providers should you discover the CDN isn’t delivering on its response time promises.

3. Implementation Convenience & Performance Analytics

Many of the more recent CDNs offer superb ease of use when configuring, and speaking as a battle hardened agency-side technical support, that goes a long way in making it into favour with your client’s technical team’s shortlist.

For all the technical sophistication possible with CDNs, the ability to largely hand over the configuration to a few simple WYSIWYG steps in a provider interface has a lot going for it in terms of implementation overhead.

It also prevents errors creeping in to the delivery path, most of which can be critical issues for an SEO’s strategy: a huge benefit.

With such providers, implementation is usually just a case of altering the relevant DNS records to point to their network, which is a 5 minute job.

Making sure the CDN implementation is performing therefore becomes the main criteria for judging if your provider is delivering for you. Look for transparent reporting on traffic location, any user-agent or ‘known threat’ filtering, IP blocking, caching performance, and page load time overview by location.

Ideally, look for integration with your existing Analytics package as this allows you to combine into your standard reporting speed and performance metrics which are business critical to an SEO’s job these days.

Google was actually advertising Quality Rater jobs in late 2004, but today the Quality Raters don’t actually work for Google; they work for contractors such as Lionbridge, Leapforce, Butler Hill and possibly others. According to Lionbridge’s Internet Assessors Program job page, it has more than 4,500 people around the world rating search results. Leapforce’s website doesn’t indicate how many are in its program, but the job listings page includes opportunities with names like “Search Engine Evaluator,” “Social Search Engine Evaluator” and “Search Quality Judge.”

The Quality Raters’ work has become more widely known over the years thanks to a couple occasions when the guideline document that Google provides as part of their work has been leaked online. (See our posts in March 2008 and October 2011.) Webmasters have also noticed unique quality rater referral strings, indicating when one of the evaluators had visited a website.

After Jennifer Ledbetter posted about the program last fall, one current Quality Rater contacted Search Engine Land wanting to explain and clarify some of what’s been written and said about the program. Since then, with a couple breaks for holidays, I’ve traded numerous emails with this person … who, in addition to working for Lionbridge as a Quality Rater, also happens to work for a US-based search marketing agency.

To help ensure that this person, whom I’ve never met, is actually a Quality Rater, I asked for some screenshots from inside the website where the rating work is done. A couple of those are inserted within the interview, and here’s an image of the rating tasks home page showing an empty task queue.

Below, we talk about the hiring process, what Quality Raters look for when they examine websites, details of the different evaluation tasks they do and much more.

Q&A With A Google Search Quality Rater

SEL: Tell me how, when and why you got started with the Quality Rater program.

Quality Rater: I first started with Lionbridge in May of 2011. I was looking for work because my then current employer had told me I was taking a pay cut, so I needed a way to add income. I began searching all the normal places for job listings and came across one on Craigslist for a Quality Rater. It sounded cool, so I sent them my resume and they got back to me the next day saying they were excited to have me and if I could just pass a few simple tests I would be hired. That was the easy part.

Did the job listing specifically mention Google?

The listing didn’t mention anything about Google but as soon as they contacted me, they said I would be doing work related to Google.

So, you knew it was Google-related. At what point did you know that you’d be rating Google’s search results?

I knew before I got hired.

One thing I think the SEO community is missing is that this program has nothing to do with SEO or rankings. What this program does is help Google refine their algorithm. For example, the Side-by-Side tasks show the results as they are next to the results with the new algorithm change in them. Google doesn’t hire these raters to rate the web; they hire them to rate how they are doing in matching users queries with the best source of information.

Let’s talk about the hiring process. There’s some kind of test. Was it difficult?

I had six days to complete both parts of the test, with the second part opening after I passed the first test.

The tests turned out to be a 24-question, essay-response theoretical test that asked questions based on a PDF they had sent me. The questions were designed to test my ability to take the rules and apply them to situations that weren’t covered in the PDF. One that I vaguely remember was about spam and what to do if the site didn’t show any signs of spam, but it gave off a spammy feeling. It was the hardest test I have ever taken (for a reference point, I’m a Literature major who has taken graduate-level courses).

Only after having passed that test did I get to take the practical exam, which had more than 140 questions. This test had actual results that I had to rate. In order to be hired, I needed to score a 90% or higher in each of the four categories (which were Vital, Useful, Relevant and Off-Topic or Useless). Ideally, these represented the actual tasks that I would receive as a rater.

What were the questions like?

To give you example of questions asked:

Query [crispy cream], English (US)
URL: http://www.treblebooster.com/

It would then be up to me to visit the page — something that I want to stress, because blogs out there have been saying that a rater can rate the page without visiting it — decide if it fits the query and then assign a rating. It really is up to the rater, but the correct answer here is Useful because of the spelling. If the user had typed “Krispy Kreme,” than this result would be off-topic, but because it is “crispy cream,” and the guitars on this page are called Crispy Cream, this could be the page the user is wanting.

There were 143 just like that. It was good times.

Do you have any direct contact with anyone at Google, or do you only communicate with Lionbridge?

I have no contact with Google; it’s only Lionbridge.

After you get hired, is there some kind of training?

After I got hired there was a weekly, two-hour webinar along with training modules to complete. It was very intense training. During the first four weeks, I was required to comment on every rating I gave. These comments were then reviewed and commented on, giving me feedback on my ratings.

At what point do you get the raters’ handbook?

I got this the moment I got hired.  It basically is just a list of tasks we perform along with examples of how to rate them.

How does Lionbridge (or Google) describe the handbook?

They refer to it as the guidelines, not a handbook.

While we are on the subject of guidelines, one thing that really impressed me was how they have more than one rater looking at a site. I believe (I’m not sure, I’m going off the comments left by other raters) that there are about six raters looking at each task. If I rate something as useful but another rater says it’s off-topic, we must come to an agreement (through comments and debate) before the rating is submitted.

How much do you make and how often do you get paid?

I get paid $14.50/hour and I am paid once a month. I’m only able to work a max of 20 hours a week and a total max of 80 hours a month.

In one of the recent articles about the Quality Raters, it says you can only work for a year and then you have to wait three months before you can re-apply. Is that true?

I know they say you can only be a rater for a year, but everyone I’ve talked to says that, as long as they get their hours in and keep up the quality they are allowed to rate.

Is the schedule completely up to you, or do they give you assigned hours?

I schedule my own hours; as long as I get at least 10 but no more than 20, I stay on pretty good terms with them. They are very strict, but allow you to make up hours that you missed. So, if I only did four hours the first week, I could make up the hours by doing 16 hours the next week. Still only allowed 20 hours a week max, so if I miss more hours than I can make up, I’m out of luck.

They also tend to be really strict about their productivity goals. There is a certain number of tasks that I must complete every minute, depending on the task type. If I fall short of those goals, I am put on probation, during which I can not work. If my quality isn’t up to par, they fire me. It’s a very controlled work environment.

You mentioned there about getting fired “if my quality isn’t up to par.” How do you know if you’re doing a good job? It seems to me that in a lot of cases, rating search results is pretty subjective.

Results are subjective, but they have a quality center that shows your progress over time. They track how many returned results you have, how long it takes you to take care of a troubled rating, etc. While the rating is up to me, it has to be similar to what other raters have said. So, they track quality based on staying within the time period for rating tasks and the number of tasks you have returned to you.

They return tasks to you — what does that mean?

It means that there has been a disagreement on the rating and you have to go back in and come to an agreement with the other raters.

So, the rating of search results is a group project. Is it difficult to come to agreement?

Sometimes it’s harder to agree with raters, especially if they haven’t read the guideline like they should or if they are just starting out. However, after enough exchanges, they have a moderator come in and choose which rating matches it best. This moderator looks at our comments and makes a decision off of that.

How often does that happen in your experience?

Not very often. Most of the time if you give your reasoning for why you rated something one way, the other raters will agree with you. Most of the time, these types of disagreements occur when something is either slightly relevant or off-topic. Once in a while, someone will think that a page is spam that isn’t, or the other way around. I’ve only had a moderator step in once.

What do you know about the moderators? Are they Lionbridge employees?

Yes, they work for Lionbridge. From what I know of them, they used to be raters and then got promoted.

Do you only look at organic results, or are you also grading ads/PPC landing pages?

We look at any type of page on the web. Most of them are organic results, but some of the tasks are geared towards more ad-related topics.

Do you remember an example of an ad-related task?

Not really. Most of what they were was placement on the page, order in which they are presented and which one would I click, etc.

Do you look at Google Places results and other Universal results, like News or Videos?

Yes, we do. I can think of many tasks where it shows the map of what a user was looking at before they typed in a query, and we are then to rate the results of that query based on the map they were looking at. We also rated news based on how current it was, how relevant it was to the query, and if it came from a trustworthy source. As for videos, we had to watch the video to determine if it was a match for the query and rate it Useful, Relevant, Slightly Relevant, or Off-topic.

That part about Maps is really interesting. So, in that task, they were putting you in the middle of some process — you’re not just doing tasks that involve standalone searches, but sometimes taking into account what has happened before? Does that also happen with other searches, too?

Almost all of the tasks given have to do with user experience. Even with just the basic searches, we are given the user’s language and location before we can rate a page. It’s not about if a page fits a query, it’s about if a user would find the page useful. The Maps queries (called local queries) are the only ones that give what the user was looking at before searching, but we are supposed to keep in mind what a user is expecting to see from that query with every task type. For example, if someone was in Seattle and typed in the query “weather,” they would find a page showing the weather in Florida slightly relevant; however, someone in Tampa would find it useful.

Aside from the collective rating that you described above, do you ever have other communication with other raters? Are there official or unofficial places where you can chat back and forth?

There are lots of places — forums and such on the Lionbridge site — where raters can talk to each other, but I never interact with them. I was always stressed getting my hours in for the week, so I didn’t have time to mingle.

Can you share a specific example of one of your recent tasks?

I can’t think of the exact URLs I rated, but the keyword was “Nike Women’s Running Shoes.” It gave me a list of 20 URLs to rate (10 on each side) [Ed. note: he's referring to the "Side-by-Side" tasks mentioned earlier.] and I visited each one in order to determine whether they were vital, useful, relevant, slightly relevant, or useless. With a recognized brand name like that, it wasn’t hard to determine quality. For example, I think the Nike site was one of the options, so that would get a “vital” rating. I remember a couple of sites sold the shoes, so I gave them a “useful” rating and the Wikipedia entry on Nike was giving a rating of “slightly relevant” because I believe not many people searching for Nike Women’s Running Shoes want a history of the company.

Do you click through and review all ten results that show up for a given task?

I always click all the links simply because I’m not good enough to tell what the site is about by just reading its description. No one is good enough, that’s why they give us the links.

When you click through from a Google search result page, what are you looking for on the web page that you visit?

When looking at a site, I always check for spam signals first — keyword stuffing, hidden text, sneaky redirects, and the like. Once I know it’s a good site, I start to look at the page as a person who would type the query in Google and whether or not the content on the page would help me fulfill my needs. There are some tasks that ask about design and layout and the like, but for the normal URL rating or Side-by-Side tasks, I really just look at content and figure out if it would be a worthwhile page for a user to see.

Do you ever look at the source code or anything like that? Are Raters asked or trained to look at source code of the web pages being rated?

There is a quick primer on looking at the source code in the guidelines, nothing in depth. Basically we look for hidden keywords and other spammy tactics discussed in the guidelines.

You mentioned URL rating tasks and Side-by-Side tasks, but also some that involve design and layout. What are those tasks like?

Design tasks ask if the page has a good ratio of main content, supplemental content, and ads. It also asks about the overall design, is it easy to read, clear communication of information, and the like. It’s not about whether the page is beautiful or amazing, but whether or not the normal user could find what they need on the page without getting lost.

Do they give you a single web page and ask you to rate its design, or are you still going through a page of search results and then rating design?

They are specific tasks, not part of rating a URL.

Are spelling and grammar part of the design-based tasks?

Spelling and grammar are something we look at in all tasks (at least I do) but there’s not a ding for it.

When looking at design and layout, do your criteria change based on the type of site you’re looking at? For example, a web page on a big brand site might be expected to have a more professional design than some small business sites.

Like I said before, it’s more about the layout than the actual design. A company with a simple design would be rated just as well as a big company with a professional design as long as the information is clear and presented in a way that is easy to understand. To give you an example, a page where you can tell what the main content is with ads taking second page in the design would get a high rating. A page where the ads are confused with the main content, where you can’t tell the difference between content and ads would get a low rating.

How many different kinds of tasks are there? The guidelines I’ve seen begin by saying “you will work on many different types of rating projects.”

There are a lot of different tasks but they are all grouped under four main groups: URL, Side-by-Side, Experimental, and Result Review.  The big one there is the Experimental tasks which have a ton of different types of tasks in them. I’ve included a picture that lists all the task types and how long they are supposed to take, as well.

What are “Display Block” and “TTR” tasks?

Display Block, if I remember right, is a block of images that we rate as a whole rather than one at a time. TTR stands for Time to Rate, which is the baseline task they use to determine how long it should take to get a task done. It has all the different tasks in it, but instead of looking for accuracy it just cares about time.

Do they try to give you tasks related to topics and things you know about, or do you review pages about things you’re not very familiar with?

If someone types in “Best Dog Food for Puppies,” it’s not very hard to know what they are wanting and most queries have a fairly obvious meaning. However, once in a while I’ll get one that I can’t figure out and that’s when I do research to figure out what they want. For example, if someone queried “Release Liner,” I would need to do some research to figure out that it’s something used in cutting vinyl for signs and the like. At that point, I could determine whether a site is worthwhile or not. Granted, it’s not a perfect system but it works most of the time.

Are there specific industries/niches that show up more than others in your rating tasks?

Not that I have noticed.

How does your work affect Google’s search results — do they tell you anything about that?

They don’t talk about that; however, I know that what it really does is perfect the algorithm instead of changing actual live search results. I gathered this from the way that Side-by-Side are the most important tasks because they show the old algorithm versus a change in the algorithm that they are testing.

Are you an active Rater these days? How long do you think you’ll keep doing it?

I still rate on the weekends. I like doing it, so I’ll keep doing as long as I can.

Does Lionbridge and/or Google know that you work in the search marketing industry?

No. I got this job after I got the Lionbridge job.

Do you know of any other search marketers who are also Quality Raters?

I don’t know any personally, but I bet there aren’t a lot of us.

What’s your opinion of Google’s search results, and has that opinion changed since you became a Quality Rater?

I’ve always used Google as my “go to” search engine; however, since I became a rater, I’ve started using it more because I can see the behind-the-scenes improvements they are trying to make.

I like the idea that they have an army of actual people working towards bettering their engine. I know some people might think this wrong or even that raters have a negative effect on their rankings. Well, I can honestly say that they don’t. The whole point behind quality raters is not to rate the actual web, but rather rate how well Google is doing at providing quality results.

Almost every company has some form of quality control. Do people get upset that McDonald’s has someone check the quality of their food? I don’t see what Google does as any different than wanting to present the best possible product they can to their users.

So, to answer your question, yes, my opinion has changed for the better.

Where We Came From

Some might believe that search engines hate SEOs, hate publishers and have done little over the years to help them. They are mistaken, either choosing to deliberately ignore the gains or, more likely, are simply unaware of how far things have come.

When I first started writing about SEO issues nearly 16 years ago, in 1996, we had little publisher support beyond add URL forms. Today, we have entire toolsets like Google Webmaster Central and Bing Webmaster Tools, along with standalone features and options, which allow and provide:

• Ability to submit & validate XML sitemaps
• Ability to view crawling & indexing errors
• Ability to create “rich” listings & manage sitelinks
• Ability to migrate a domain
• Ability to indicate a canonical URL or preferred domain
• Ability to set crawl rates
• Ability to manage URL parameters
• Ability to view detailed linkage information to your site
• Ability to view keywords used to reach your site
• Notifications of malware or spam issues with your site

There’s even more beyond what I’ve listed above. The support publishers enjoy today was simply unimaginable to many veteran SEOs who were working in the space a decade ago.

The advancement has been welcomed. It has helped publishers better manage their placement in those important venues of the web, the search engines. It has helped search engines with errors and problems that would hurt their usability and relevancy.

That’s why 2011 was so alarming to me. After years of moving forward, the search engines took a big step back.

The Loss Of Link Data

One of the most important ways that search engines determine the relevancy of a web page is through link analysis. This means examining who links to a page and what the text of the link — the anchor text — says about the page.

However, for years Google has deliberately suppressed the ability for outsiders to see what links tell it about any particular page. Want to know why THAT result shows up for Santorum? Why Google was returning THAT result for “define English person” searches? Sorry.

Google won’t help you understand how links have caused these things. It refuses to show all the links to a particular page, or the words used within those links to describe a page, unless you are the page’s owner.

Why? Google’s rationale has been that providing this information would make it harder for it to fight spam. Potentially, bad actors might figure out some killer linking strategy by using Google’s own link reporting against it.

It’s a poor argument. Despite withholding link data, it’s painfully easy to demonstrate how sites can gain good rankings in Google for competitive terms such as “SEO” itself by simply dropping links into forums, onto client pages or into blog templates.

Given this, it’s hard to understand what Google thinks it’s really protecting by concealing the data. But until 2011, there was an easy alternative. Publishers and others could turn to Google-rival Yahoo to discover how people might be linking to a page.

Goodbye Yahoo Site Explorer

Yahoo launched its “Yahoo Site Explorer” back in September 2005, both as part as a publicity push to win people away from Google and to provide information to publishers. The tool allowed anyone to see what link data Yahoo had about any page in its listings.

Today, Yahoo still supposedly wants to win people away from Google. But because Yahoo’s web search results are now powered by Bing, Yahoo has little reason to provide tools to support publishers. That’s effectively Bing’s problem now.

Yahoo closed Yahoo Site Explorer at the end of last November, saying as it still does on the site now:

Yahoo! Search has merged Site Explorer into Bing Webmaster Tools. Webmasters should now be using the Bing Webmaster Tools to ensure that their websites continue to get high quality organic search traffic from Bing and Yahoo!.

That’s not true. Yahoo Site Explorer was not merged into Bing Webmaster Tools. It was simply closed. Bing Webmaster Tools doesn’t provide the ability to check on the backlinks to any page in the way that Yahoo Site Explorer allowed.

The closure supposedly came after Yahoo “listened to your feedback” about what publishers wanted, as it posted earlier this year. I don’t know what feedback Yahoo was hearing, but what I’ve heard has been people desperately pleading with Yahoo or Bing to maintain the same exact features that Yahoo Site Explorer provided — and pleading for well over a year.

Yahoo-Bing Deal Has Reduced Competition & Features

When the US Department Of Justice granted its approval for Yahoo to partner with Microsoft, that was supposed to ensure that the search space stayed competitive. From what the Department Of Justice said in 2010:

After a thorough review of the evidence, the division has determined that the proposed transaction is not likely to substantially lessen competition in the United States, and therefore is not likely to harm the users of Internet search, paid search advertisers, Internet publishers, or distributors of search and paid search advertising technology.

I’d say dropping Yahoo Site Explorer did harm to both users of internet search and internet publishers. Yahoo Site Explorer was a distinctive tool that only Yahoo offered, allowing both parties named by the DOJ to better understand the inner workings of the search engines they depend on. It also reduced competitive pressure for Google to offer its own tool.

Indeed, things have gotten worse since Yahoo Site Explorer closed. At the end of last December, Bing officially confirmed that it no longer supports the link command in its help forum.

Next To Go, The Link Command?

The link command allows you to enter any page’s web address prefaced by “link:” in order to find links that point at that page. It’s a long-standing command that has worked for many major search engines as far back to late 1995, when AltaVista launched.

Google still supports this command to show some (but not all) of the links it knows about that point at pages. I’d link to Google’s documentation of this, but the company quietly dropped that some time around May 2008. Here’s what it used to say:

Here’s how the command still works at Google. Below, I used it to see what links Google says point to the home page of the official Rick Santorum campaign web site:

The first arrow shows you how the command is being used. The second arrow shows you how Google is reporting there are 111 links pointing to the page. Imagine that. Rick Santorum, currently a major Republican candidate for US president, and Google says only 111 other pages link to his web site’s home page.

The reality is that many more pages probably link over. Google’s counting them but not showing the total number to people who care about what exactly is being considered. I’ll demonstrate this more in a moment, but look at the worse situation on Bing:

One link. That’s all Bing reports that it knows about to those in the general public who may care to discover how many links are pointing to the Rick Santorum web site.

It’s Not Just An SEO Thing

People do care, believe me. I actually started writing this article last Monday and got interrupted when I had to cover how Google might have been involved with a link buying scheme to help its Chrome browser rank better in Google’s own search results.

I doubted that was really the main intent of the marketing campaign that Google authorized (Google did err on the side of caution and punished itself), but the lack of decent link reporting tools from Google itself left me unable to fully assess this as an independent third-party.

As soon as that story was over, renewed attention was focused on why Rick Santorum’s campaign web site wasn’t outranking a long-standing anti-Santorum web site that defines “santorum” as a by-product of anal sex.

Major media outlets were all over that story. My analysis was cited by The Economist, CNN, The Telegraph, The New York Times, MSNBC and Marketplace, to name only some.

But again, I — or anyone who really cared — was unable to see the full links that Google knew about pointing at both sites, much less the crucial anchor text that people were using to describe those sites. Only Google really knew what Google knew.

Third Party Options Good But Not The Solution

If you haven’t heard more complaints over the closure of Yahoo Site Explorer, and the pullback on link data in general, that’s because there are third-party alternatives such as Majestic Site Explorer or the tool I often use, SEOmoz’s Open Site Explorer.

These tools highlight just how little the search engines themselves show you. Consider this backlink report from Open Site Explorer for the Rick Santorum campaign’s home page:

The first arrow shows how 3,581 links are seen pointing at the page. Remember Google, reporting only 111? Or Bing, reporting only 1?

The next two arrows show the “external” links pointing at both the Santorum home page and the anti-Santorum home page. These are links from outsiders, pointing at each page. You can see that the anti-Santorum page has four times as many links pointing at it than the Santorum campaign page, a clue as to why it does so much better for a search on “santorum.”

But it’s not just number of links. Using other reports, I can see that thousands of links leading to both sites have the text “santorum” in the links themselves, which is why they both are in the top results for that word.

Because the anti-site has so many more links that say “santorum” and “spreading santorum,” that probably helps it outrank the campaign site on the single word. But because the official site has a healthy number from sources including places like the BBC saying “rick santorum” in the links, that — along with its domain name of ricksantorum.com — might help it rank better for “rick santorum.”

It’s nice that I can use a third party tool to perform this type of analysis, but I shouldn’t have to. It’s simply crazy — and wrong — that both Google and Bing send searchers and publishers away from their own search engines to understand this.

For one, the third party tools don’t actually know exactly what the search engines themselves are counting as links. They’re making their own estimates based on their own crawls of the web, but that doesn’t exactly match what Google and Bing  know (though it can be pretty good).

Not Listing Links Is Like Not Listing Ingredients

For another, the search engines should simply be telling people directly what they count. Links are a core part of the “ingredients” used to create the search engine’s results. If someone wants to know if those search results are healthy eating, then the ingredients should be shared.

Yes, Google and Bing will both report link data about a publisher’s own registered site. But it’s time for both of them to let anyone look up link data about any site.

The Blekko search engine does this, allowing anyone logged in to see the backlinks to a listed page. Heck, Blekko will even give you a badge you can place on your page to show off your links, just as Yahoo used to. But for Google, it’s “normal” for its link command to not show all the links to a page. Seriously, that’s what Google’s help page says.

Google, in particular, has made much of wanting people to report spam found in its search results. If it really wants that type of help, then it needs to ensure SEOs have better tools to diagnose the spam. That means providing link data for any URL, along with anchor text reporting.

Google has also made much about the need for companies to be open, in particular pushing for the idea that social connection should be visible. Google has wanted that, because until Google+ was launched, Google had a tough time seeing the type of social connections that Facebook knew about.

Links are effectively the social connections that Google measures between pages. If social connections should be shared with the world, then Google should be sharing link connections too, rather than coming off as hypocritical.

Finally, it doesn’t matter if only a tiny number of Google or Bing users want to do this type of link analysis. That’s often the pushback when this issue comes up, that so few do these type of requests.

Relatively few people might read the ingredients labels on the food they eat. But for the few that do, or for anyone who suddenly decides they want to know more, that label should be provided. So, too, should Google and Bing provide link data about any site.

Goodbye Keyword Referrer Data

While I’m concerned about the pullback on link data, I’m more concerned about how last October, Google stopped reporting to publishers the keywords people used to find their web sites, for times when those people were logged into Google.

Link data has long been suppressed by Google. Holding back on keyword data is a new encroachment.

Google has said this was done to protect user privacy. I have no doubt many in the company honestly believe this. But it if was really meant to protect privacy, then Google shouldn’t have deliberately left open a giant hole that continues to provide this data to its paid advertisers.

Worse, if Google were really serious about protecting the privacy of search terms, then it would disable the passing of referrers in its Chrome browser. That hasn’t happened.

Unlike the long examination of link data above, I’ll be far more brief about the situation with Google withholding link data. That’s because I’ve already written over 3,000 words looking at the situation in depth last October, and that still holds up. So please see my previous article, Google Puts A Price On Privacy, to understand more.

Google’s Weak Defense

Since my October story, the best defense that Google’s been able to concoct for withholding keyword data from non-advertisers is a convoluted, far-fetched argument that makes its case worse, not better.

Google says that potentially, advertisers might buy ads for so many different keywords that even if referrer data was also blocked for them, the advertisers could still learn what terms were searched for by looking through their AdWords campaign records.

For example, let’s say someone did a search on Google for “Travenor Johannisoon income tax evasion settlement.” I’ve made this up. As I write this, there are no web pages matching a Google search for “Travenor Johannisoon” at all. But…

• If this were a real person, and
• someone did that search, and
• if a page appeared in Google’s results, and
• someone clicked on that page…

then the search terms would be passed along to the web site hosting the page.

Potentially, this could reveal to a publisher looking at their web analytics that there might be a settlement for income tax evasion for involving a “Travenor Johannisoon.” If the publisher starting poking around, perhaps they might uncover this type of information.

Of course, it could be that there is no such settlement at all. Maybe it’s just a rumor. Anyone can search for anything which doesn’t make it into a fact.

More likely, the search terms are so buried in all the web analytics data that the site normally receives that this particular search isn’t noticed at all, much less investigated.

Extra Safe Isn’t Extra Safe

Still, to be extra safe, Google has stopped passing along keyword data when people are signed-in. Stopped, except to its advertisers. Like I said, Google argues that potentially advertisers might still see this information even if they were also blocked.

For instance, say someone runs an ad matching any searches with “income tax evasion” in them. If someone clicked on the ad after doing a search for “Travenor Johannisoon income tax evasion settlement,” those terms would be passed along though the AdWords system to the advertiser, even though the referrer might pass nothing to the advertiser’s web analytics system.

So, why bother blocking?

Yes, this could happen. But if the point is to make things more private, then blocking referrers for both advertisers and non-advertisers would still make things harder. Indeed, Google still has other “holes” where “Travenor Johannisoon” might find his privacy exposed just as happens potentially with AdWords.

For example, if someone did enough searches on the topic of Travenor and tax evasion, that might cause it to appear one of Google Instant’s suggested searches.

So why bother blocking?

Also, while Google blocks search terms from logged-in users in referrer data, those same searches are not blocked from the keyword data it reports to publishers through Google Webmaster Central. That means the Travenor search terms could show up there.

So why bother blocking?

Nothing has changed my view that, despite Google’s good intentions, its policy of blocking referrers only for non-advertisers is incredibly hypocritical. Google purports this is done to protect privacy, but it puts its own needs and advertisers desires above privacy.

Blocking referrers is a completely separate issue from encrypting the search results themselves. That’s good and should be continued. But Google is deliberately breaking how such encryption works to pass along referrer data to its advertisers. Instead, Google should block them for everyone or block them for no one. Don’t play favorites with your advertisers.

What Google & Bing Should Do

Made it this far? Then here’s the recap and action items for moving forward.

Bing should restore its link command, if not create a new Bing Site Explorer. Google should make sure that its link command reports links fully and consider its own version of a Google Site Explorer. With both, the ability for anchor text reports about any site is a must.

If there are concerns about scraping or server load, make these tools you can only use when logged in. But Yahoo managed to provide such a tool. Blekko is providing such statistics. Tiny third-party companies are doing it. The major search engines can handle it.

As for the referrer data, Google needs to immediately expand the amount of data that Google Webmaster Central reports. Currently, up to 10,000 terms (Google says up to 1,000, but we believe that’s wrong) for the past 30 days are shown.

In November, the head of Google’s spam team Matt Cutts — who’s also been involved with the encryption process — said at the Pubcon conference that Google is considering expanding the time period to 60 days or the queries to 2,000 (as said, we think — heck, we can see, they already provide more than this). Slightly more people wanted more time than more keywords shown.

I think Google should do more than 60 days. I think it should be providing continuous reporting and holding that data historically on behalf of sites, if it’s going to block referrers. Google is already destroying historical benchmarks that publishers have maintained. Google’s already allowed data to be lost for those publishers, because they didn’t begin to go in each day and download the latest information.

So far, all Google’s done is provide an Python script to make downloading easier. That’s not enough. Google should provide historical data, covering a big chunk of the terms that a site receives. It’s the right thing to do, and it should have been done already.

What Publishers Can Do

An anti-SOPA-like effort as targeted GoDaddy isn’t going to work with the search engines. That’s because the two biggest things that publishers could “transfer” out of Google and Bing are their ads and their web sites. But there’s no place to transfer these to that wouldn’t hurt the publishers with incredible amounts of lost traffic.

This doesn’t mean that publishers are powerless, however.

Bing is desperate to be seen as the “good” search engine against “evil” Google. Publishers should, whenever relevant, remind Bing that it’s pretty evil not to have maintained its own version of Yahoo Site Explorer much less to have closed the link command.

Mention it in blog posts. Mention it in tweets. Bring it up at conferences. Don’t let it die. Ask Bing why it can’t do what little Blekko can.

As for Google, pressure over link data is probably best expressed in terms of relevancy. Why is Google deliberately preventing this type of information from being studied? Is it more afraid that doing so will reveal weaknesses in its relevancy, rather than potential spam issues? Change the debate to relevancy, and that gets Google’s attention — plus the attention of non-publishers.

There’s also the issue of openness. Google shouldn’t be allowed to preach being “open” selectively, staying closed when it suits Google, without some really good arguments for remaining closed. On withholding link data, those “closed” arguments no longer stand up.

As for the referrer data, Google should be challenged in three ways.

First, the FTC will be talking to publishers as part of its anti-trust investgation into Google’s business practices. Publishers, if asked, should note that by withholding referrer data except for Google’s advertisers, it’s potentially harming competing retargeting services that publishers might prefer to use. Anti-trust allegations seem to really get Google’s attention, so make that wheel squeak.

Second, question why Google is deliberately leaving a privacy hole open for the searchers it’s supposedly trying to protect. If Google’s really worried about what search terms reveal, the company needs a systematic way to scrub potentially revealing queries from everything: suggested searches, reporting in Google Webmaster Central, AdWords reporting as well as referrer data.

Finally, withhold your own data. Are you opted-in to the data sharing on Google Analytics that launched back in 2008? Consider opting-out, if so:

To opt-out, when you log in, select an account, then select “Edit Analytics Account” next to the name of the account in the Overview window, then you’ll see options as shown above and as explained on this help page.

Opting out means you can’t use the benchmarking feature (fair enough, and no loss if you don’t use it) and Conversion Optimizer. If you still want Conversion Optimizer, don’t opt-out or alternatively, tell Google that you should have a choice to share data solely for use with that product but not other Google products.

There might be other drawbacks to not sharing that I’m missing. But we haven’t been sharing here at Search Engine Land since the beginning of the year. So far, we’re not having any problems.

Google loves data. Withholding your own is another way for publishers to register their displeasure about having data withheld from them. And it’s the type of thing that Google just might notice.

Related Articles

• What Is SEO / Search Engine Optimization?
• Search Engine Land’s Guide To SEO
• The Periodic Table Of SEO Ranking Factors
• What Is Google PageRank? A Guide For Searchers & Webmasters
• How The “Focus On First” Helps Hide Google’s Relevancy Problems
• Yahoo Completes Global Organic Transition To Bing
• Yahoo Site Explorer Closing Down Monday, November 21st
• The Microsoft-Yahoo Search Deal, In Simple Terms
• Google Releases New Link Reporting Tools
• Google Now Reporting Anchor Text Phrases
• Yahoo Adds Link Badge In Site Explorer
• Link Building Tool Review: Link Research Tool Set By CEMPER
• Link Building Tool Review: SEOmoz PRO
• Link Building Tool Review: SEO Book
• Link Building Tool Review: Raven Tools
• Link Building Tool Review: Majestic SEO
• Bing Webmaster Tools Launches New Link Reports; Google Webmaster Tools Changes Theirs
• Blekko’s SEO Tools: What Information Do They Provide?
• Blekko Offers New Linkroll Widget & More Publisher Tools
• Google’s Chrome Page No Longer Ranks For “Browser” After Sponsored Post Penalty
• Should Rick Santorum’s “Google Problem” Be Fixed?
• How Rick Santorum Is Making His “Google Problem” Worse
• For “Define An English Person,” Google Suggests The C-Word
• Google’s Spam Report Page Gets “Biggest Refresh” In Years
• Google: As Open As It Wants To Be
• Google’s Facebook Competitor, The Google+ Social Network, Finally Arrives
• How Google Instant’s Autocomplete Suggestions Work
• Google Webmaster Tools Adds Page-Level Query Data
• Google Webmaster Tools Search Queries Report Now Less Accurate
• Google Analytics Benchmarking Feature, Data Sharing & Audio Ad Charting
• Guest Opinion: Is Google’s Privacy Move Really An Anti-Competitive Practice?
• Dear Congress: It’s Not OK Not To Know How Search Engines Work, Either
• Googleopoly: The Definitive Guide To Antitrust Investigations Against Google
• Keyword “Not Provided” By Google Spikes, Now 7-14% In Cases
• Google Puts A Price On Privacy

The question asked was: “If you had to put all your marketing time and budget into only one channel, what would it be?” The list of choices included SEO, paid search, mobile, social and traditional media. As you can see below SEO beats everything else by a mile.

What’s also interesting is how high “traditional media” ranked. That category would include newspapers, yellow pages, direct mail, traditional radio and so on. Newspapers and direct mail were the top traditional channels being used by this group of survey respondents.

However Facebook (not Facebook Ads) was the most common marketing tool, used by 70 percent of these SMB respondents. The question was, “Are you promoting your business with the following websites/services?”

The online survey was conducted in November. The sample came from MerchantCircle’s 1.6 million SMB member database. Just under 80 percent of respondents were businesses with fewer than four employees. The majority (63 percent) had annual marketing budgets of $2,500 or less.

There are additional findings about the use of social media, deals and mobile marketing that I discuss on my personal blog Screenwerk.

Before There Was Panda, There Was Florida

At the end of 2003, Google released what became known as its Florida Update. Now kids, if you think Panda was bad, Florida was a storm that seemingly blew the roofs off SEO homes everywhere.

It hit during the holiday shopping period. It caused confusion and havoc among many SEOs. Search engine optimization forums like Webmaster World, which gave Florida its name, lit up like crazy.

As with the Panda Update, there were plenty of people who actually did better after Florida than before. With any update, for all the losers that yell loudly about their pain, there are winners who silently smile over their fortune.

Still, there were lots of publishers that felt they’d been doing right by Google but then penalized for reasons they didn’t understand. And it came out of the blue, something they didn’t expect.

How About Some Warning?

As the industry cleaned up from the Florida mess, there was much discussion on how the search engines and SEOs could better work together.

While at times they can seem enemies, they are also part of the same ecosystem. Indeed, Google itself recommends that people follow SEO best practices.

I gave a keynote talk in August 2004, where I suggested a variety of ideas I’d gathered from an online discussion. One of these was from Nuttakorn Rattanachaisit, who said:

SEs should pay attention to webmaster or website owner by giving them the information when they are changing the algorithms or something which are affected to their site. Because they don’t know what happen about their site when they dropped from the listing. Sometimes it is not their fault. I see many website which has a lot of quality content but they aren’t ranked in Search engine.

I termed this a “weather report” and listed it at the top of ideas I discussed. The slide above is from my talk back then.

2005: Yahoo Issues The First Weather Report, Google Follows

On March 31, 2005, Yahoo granted my wish — and Rattanachaisit’s suggestion — by issuing its first algorithmic weather report. I got a nice call-out in Yahoo’s post, but more important, publishers across the web got a pledge that Yahoo would be keeping them updated about coming changes:

It is important to have a dialog with publishers who rely on our product to send them visitors and that these content providers provide us with valuable feedback on our search products.

So consider this our first weather report. We’re releasing a new index tonight. You should see a lot of new content in the index as well as fluctuations in the rankings of results from previous searches.

Google soon followed with its own weather reports. The first that I can track down was issued by Cutts on August 19, 2005, where he wrote:

In fact, I’ll issue a small weather report: I would not recommend using sneaky JavaScript redirects. Your domains might get rained on in the near future.

Weather Reports Die Out Toward 2010

The weather reports kept coming, in various ways, through early 2010. Here, you can see some of the many that Yahoo issued:

As for Google, weather reports continued to come unofficially through Matt’s blog, such as this one in 2006, in 2007, or a psuedo-weather report about new Toolbar PageRank coming out in 2008.

I’m not sure when the Google reports ended. Perhaps they never really have, as I know Cutts has continued to offer tons of unofficial heads-ups about various things through his blog. But certainly, I think it’s fair to say that the idea of search engines issuing regular weather reports kind of died off.

Bringing Back The Weather Reports

Perhaps good ideas always get reinvented. Two weeks ago, during the US Senate hearing about Google and competition, Senator Amy Klobuchar had one of the last questions to put to Google chairman Eric Schmidt. And what she wanted was weather reports. From my live blogging (the brackets show me exclaiming that hey, she wants weather reports!):

Klobuchar: Can you do more to explain when changes are coming [weather reports!]

Schmidt: We do some, we could do more. but if we’re completely transparent, we’d be heavily gamed by sites trying to spam us. Do believe we can do better job to describe the change.

After last week’s news about the Panda update was made official, I went back to Google and asked about bringing back the weather report concept. That’s because, despite what Schmidt said, lately Google hasn’t been issuing warnings before a change happens.

Google will confirm some updates if we or other publications ask, but this is after the fact. And we’d rather not be in the position of having to constantly be asking if Google would confirm if something that just happened was a Panda update, or another type of algorithm update or so on. Having regular official updates about updates (heh) through something like the Google Webmaster Central blog would be better.

In response, Google told me last week:

We’ve been reserving the moments to blog about changes when it’s a significant one.

Panda Weather Report Issued

That seemed pretty much like no, there aren’t going to be weather reports. But then early today, Matt Cutts tweeted:

Weather report: expect some Panda-related flux in the next few weeks, but will have less impact than previous updates (~2%).

Panda doesn’t constantly run. It’s a special algorithm that Google processes content through on a periodic basis. Why Google Panda Is More A Ranking Factor Than Algorithm Update explains much more about this.

What Cutts is saying isn’t that Panda “flux” will be happening during the next few weeks but rather to expect another Panda Update (Panda 2.6), a minor one compared to some past ones, to happen in the next few weeks.

That fits in with the regular cycle we’ve come to know, which so far has been this:

• Panda Update 1.0: Feb. 24, 2011
• Panda Update 2.0: April 11, 2011 (about 7 week gap)
• Panda Update 2.1: May 10, 2011 (about  4 week gap)
• Panda Update 2.2: June 16, 2011 (about 5 week gap)
• Panda Update 2.3: July 23, 2011 (about 5 week gap)
• Panda Update 2.4: August 12, 2011 (about 3 week gap)
• Panda Update 2.5: September 28, 2011 (about 7 week gap)

Overall, I’m encouraged. Weather reports might be making a comeback, and that’s a good thing. I’ve asked Google again about this, and I’ll update if I hear more. And Bing, if you’re listening, we want weather reports from you, too.

Postscript: I received this further update from Google:

In general, we still want to be giving weather reports or advance notice where it makes sense. It can be difficult to decide when to talk about updates or algorithm changes, not only because changes are often not very noticeable to the general populace, but because we make over 500 algorithmic changes to our rankings a year.

All of the examples below are great ideas that any link builder can use at an agency or in-house. In my experience, my business (empirecovers.com) has shown to benefit from each of these tactics and we’ll touch on each in the following.

Empire Covers has consistently targeted keywords such as boat covers, motorcycle covers and car covers, and ever since we moved link building in-house, we’ve been in the top 5 for all keywords we target.

If you know anything about Search Engine Optimization, you know there is a correlation between building text links and higher rankings. These are the steps we go through in order to build links.

The first thing we do is search through Twitter to find people with active followings and who are also engaged in the blogging community. We follow these Twitter users because they seem to be more willing to respond to tweets.

We also send mass emails every month in efforts to get people to review our product, but we have actually found Twitter is a much better tool in reaching out to people.

Also, Twitter seems to work faster in a sense, since people respond to a question much faster on the Twitter platform than via email or even Facebook.

Here are some more things to think about when engaging people:

1. Try to enter or sponsor any contests they may be having.
2. Do not talk about links or text links right off the bat. Try to engage them in their company or something that they are interested in.
3. Even if it’s a YouTube video and not a text link, be happy with what you get. I have used many of the YouTube videos that people have produced for me  on our site and it is a great for the social side.

Using Product Reviews To Get Links

The most valuable and effective way to build links is to offer to send them a free sample of your product to review. I’ll note here that I was once at a conference and talked to someone from Bing that was in charge of their spam and SEO, he explained product reviews are a sort of grey area, but in my mind, any grey area is a green light.

Product reviews are an easy way to obtain publicity and feedback regarding your product or brand. We have averaged around 40 to 60 quality links per month via reviews and that seems to be working well for us.

Here are a few of my favorite example reviews and contest sponsorship links which we’ve obtained in this manner.

Video Review

• http://www.rvwithtito.com/2011/09/i-dont-have-rv-garage-i-have-rv-cover.html

This guy was extremely happy to find an RV cover that fit his abnormally large RV. The video he created was just the icing on the cake. Also, notice how we strategically asked him to add RV cover and empirecovers.com to the video. All in all, he created excellent review for us and gave us an SEO boost to boot.

Simple Step-by-Step Review

• http://aquaskier.com/reviews/empirecovers_silvershark.htm

This is a great review for a boat cover that just happens to be very interesting.

Contest Sponsorship

• http://www.travelwithkevinandruth.com/2011/05/contest-giveaway.html

This is a contest we sponsored last month for an RV cover, which generated a lot of community interest as well, since  RV people love to comment on links. It was extremely cool to do, but honestly, also extremely hard to put on, taking three people to manage the contest.

Kids Demo How To Use Product on YouTube

• http://www.youtube.com/watch?v=q5RteIc1aTU&feature=player_embedded

You’d likely have to pay an agency top dollar for that review. Everyone in the office and the Web loved it and we got a lot of hits because of it.

Blogger Creates Social Proof

• http://half-throttle.com/post/6039003719/this-cover-is-from-empire-covers-i-have-the

This is a review we absolutely love and use on our site. This guy lived in Panama during the monsoon season and used our covers. Once we put up the video on our motorcycle page, our conversion rate went up by about .5% to 1%, so from that perspective alone, it was pretty amazing.

I believe this method of link building is beneficial in many ways, since it’s good for the customers and also helps with everyday content on your site. This stuff is priceless when it comes to free advertising, and if a cover company can do it, why can’t you?

The survey tackles issues such as on-page factors (keyword usage in headlines, keyword usage in page titles, etc.), off-page factors like inbound links from other websites and social media exposure, time-based factors, publisher authority and much more. It’s quite comprehensive and anyone publishing news and looking for traffic from Google News should find plenty to think about in the survey results.

Here’s a peek at what the 17 contributors say are the top overall Google News ranking factors:

The survey contributors have worked on some of the web’s largest and most well-known news sites, including the New York Times, ABC News, ESPN, CNN, Huffington Post and others.

The organizers say that their plan is to do the survey again next year, hopefully with a larger group of survey takers.

I’m really pleased with how the video turned out, and I hope you are, too. We worked with Common Craft to produce it, and they have many more great explainer videos like this in their Common Craft video library, so check that out!

You’ll find our video on YouTube in our Search Engine Land channel, and it also has a permanent home here on Search Engine Land as part of our What Is SEO / Search Engine Optimization? page.

If you haven’t visited that page, do check it out. It also provides background our about our more in-depth Periodic Table Of SEO Ranking Factors, links to free guides about SEO, resources here at Search Engine Land and across the web on the topics.

Hey, if you like that page, links and social shares to it are appreciated!

By the way, if you’re a Search Engine Land member, don’t forget a new feature we’ve added — our Search Engine Land Guide To SEO as a downloadable eBook. It’s just one of many benefits you get. Not a member? Consider signing-up!

There are two major scenarios that we will examine, including a look at the potential solutions and the choices that publishers can make.

No Place For Rel=Canonical

The “The Really Complicated Technical SEO Infrastructure Issues” panel at SMX Advanced started with controversy when REI’s Jonathan Colman said that REI.com benefitted from using rel=canonical on the product pages of its catalog.

For example, if there were 10 pages of tent products, pages 2 through 10 all implemented a canonical tag which pointed back to page 1.

This is when Google’s Maile Ohye, who was also on the panel, piped up and said that this was not a proper implementation of the canonical tag. So ideally, do not use this approach. Even though REI thinks it is working for them currently, and it might be, there is simply no assurance that it will work that way in the future.

Search engines implement their algorithms and update them from time to time based on the way they believe things are supposed to work, and any time you use a feature such as rel=canonical in a way other than intended, you face material risk of a problem at some point.

The rel=canonical tag should only be used when the target page (the one that the tag points to) has substantially all of the content on the source page (the one implementing the tag).

Scenarios where this works can occur when you have different sort orders for products, or pages that show subsets of the products on the target page.

The Scenario: Article Pagination

This scenario arises when a magazine has a very long article. One possible approach to this is simply to have a “more” button which populates the content on the existing URL if the user wants to see it (Salon Magazine uses this approach).

However, there are some publishers that choose to continue the content on one or more additional pages as you can see with this example from Discover Magazine:

In this particular example, the content for the article lives on these two different URLs:

• http://discovermagazine.com/2011/mar/09-vital-signs-those-who-know-us-best
• http://discovermagazine.com/2011/mar/09-vital-signs-those-who-know-us-best/article_view?b_start:int=1&-C=

This scenario could lead to potential SEO issues. There are two major ways to approach this situation:

1. Do Nothing

In other words, let the search engines find and index both pages. There is a strong argument for this, as the pages have different content and some searchers may only be interested in the content on the 2nd page of the article. To get the most of this strategy, consider some of the tips by Tedster on Webmaster World for optimizing paginated articles.

2. NoIndex the additional pages

You should consider this option if you believe that having visitors land on the 2nd page of the article (and other pages other than page 1) would be a bad user experience. Note that if third party publishers link to your 2nd article page, that it can still pass link juice back into the rest of your site, and that is a good thing.

The Scenario: E-Commerce Catalog Pagination

Anyone who has done shopping for popular products online has seen this scenario.

Here is an example from the Men’s Boots page on Zappos.com:

The major choices remain the same, but the arguments about them are a bit different:

1. Do Nothing

On the SEO Infrastructure panel, Maile Ohye argued that the search engines still see value on those pages because they list different products than the first page of products. The basic argument for leaving the pages alone is that they are not duplicate content and they may pick up their own search traffic.

2. NoIndex the additional pages

The counter argument to the do nothing scenario is that the unique text content on the pages is likely to be quite low. For that reason, it is not unreasonable to be concerned that the search engines will perceive these as low quality pages, and that they may therefore have a negative impact on search traffic.

Obviously, the latter would not be the goal of the search engines, but there are software algorithms involved, and these algorithms deal with a nearly limitless number of scenarios.

In spite of their best efforts, sometimes there are sites that get hurt for unintended reasons. Given that pages that simply list a bunch of products are not that likely to garner much direct search traffic it may make sense to avoid that risk.

No Place For NoFollow

If you do decide to NoIndex pages on your site, I do not advise that you NoFollow the links to those pages. NoFollow does not conserve any link juice when you implement it. It just does not pass the link juice to the page receiving the link, and that link juice is thrown away (it is not redistributed to other pages on your site).

Let the juice flow to your pages with NoIndex tags, because those pages can accumulate and pass PageRank, and they can then vote a portion of that PageRank back into the rest of your site.

Key Considerations Regarding Pagination

My inclination in the Article Pagination scenario is to simply do nothing, and let the search engines discover and index pages in the article beyond the first page.

There is little downside, and with a little page design effort, you can pretty much eliminate any concerns that users will have a poor experience landing on the 2nd page of an article.

Whether or not to place NoIndex tags on additional pages on e-commerce sites is a judgment call. Just be aware that the search engine’s preference is to discover and index that content. They want to be aware of it, and they want to handle it properly.

That said, it is not a webmaster guidelines violation to NoIndex the pages, nor is it an improper use of the NoIndex tag. In addition, there is some risk that the search engines will make an unintended mistake in understanding the value and uniqueness of the additional product pages. You need to decide which path makes you more comfortable.

1. Commit yourself to the process. SEO isn’t a one-time event. Search engine algorithms change regularly, so the tactics that worked last year may not work this year. SEO requires a long-term outlook and commitment.

2. Be patient. SEO isn’t about instant gratification. Results often take months to see, and this is especially true the smaller you are, and the newer you are to doing business online.

3. Ask a lot of questions when hiring an SEO company. It’s your job to know what kind of tactics the company uses. Ask for specifics. Ask if there are any risks involved. Then get online yourself and do your own research—about the company, about the tactics they discussed, and so forth.

4. Become a student of SEO. If you’re taking the do-it-yourself route, you’ll have to become a student of SEO and learn as much as you can. Luckily for you, there are plenty of great web resources (like Search Engine Land) and several terrific books you can read. (Yes, actual printed books!) See our What Is SEO page for a variety of articles, books and resources.

5. Have web analytics in place at the start. You should have clearly defined goals for your SEO efforts, and you’ll need web analytics software in place so you can track what’s working and what’s not.

6. Build a great web site. I’m sure you want to show up on the first page of results. Ask yourself, “Is my site really one of the 10 best sites in the world on this topic?” Be honest. If it’s not, make it better.

7. Include a site map page. Spiders can’t index pages that can’t be crawled. A site map will help spiders find all the important pages on your site, and help the spider understand your site’s hierarchy. This is especially helpful if your site has a hard-to-crawl navigation menu. If your site is large, make several site map pages. Keep each one to less than 100 links. I tell clients 75 is the max to be safe.

8. Make SEO-friendly URLs. Use keywords in your URLs and file names, such as yourdomain.com/red-widgets.html. Don’t overdo it, though. A file with 3+ hyphens tends to look spammy and users may be hesitant to click on it. Related bonus tip: Use hyphens in URLs and file names, not underscores. Hyphens are treated as a “space,” while underscores are not.

9. Do keyword research at the start of the project. If you’re on a tight budget, use the free versions of Keyword Discovery or WordTracker, both of which also have more powerful paid versions. Ignore the numbers these tools show; what’s important is the relative volume of one keyword to another. Another good free tool is Google’s AdWords Keyword Tool, which doesn’t show exact numbers.

10. Open up a PPC account. Whether it’s Google’s AdWords, Microsoft adCenter or something else, this is a great way to get actual search volume for your keywords. Yes, it costs money, but if you have the budget it’s worth the investment. It’s also the solution if you didn’t like the “Be patient” suggestion above and are looking for instant visibility.

11. Use a unique and relevant title and meta description on every page. The page title is the single most important on-page SEO factor. It’s rare to rank highly for a primary term (2-3 words) without that term being part of the page title. The meta description tag won’t help you rank, but it will often appear as the text snippet below your listing, so it should include the relevant keyword(s) and be written so as to encourage searchers to click on your listing. Related bonus tip: You can ignore the Keywords meta tag, as no major search engine today supports it.

12. Write for users first. Google, Yahoo, etc., have pretty powerful bots crawling the web, but to my knowledge these bots have never bought anything online, signed up for a newsletter, or picked up the phone to call about your services. Humans do those things, so write your page copy with humans in mind. Yes, you need keywords in the text, but don’t stuff each page like a Thanksgiving turkey. Keep it readable.

13. Create great, unique content. This is important for everyone, but it’s a particular challenge for online retailers. If you’re selling the same widget that 50 other retailers are selling, and everyone is using the boilerplate descriptions from the manufacturer, this is a great opportunity. Write your own product descriptions, using the keyword research you did earlier (see #9 above) to target actual words searchers use, and make product pages that blow the competition away. Plus, retailer or not, great content is a great way to get inbound links.

14. Use your keywords as anchor text when linking internally. Anchor text helps tells spiders what the linked-to page is about. Links that say “click here” do nothing for your search engine visibility.

15. Build links intelligently. Begin with foundational links like trusted directories. (Yahoo and DMOZ are often cited as examples, but don’t waste time worrying about DMOZ submission. Submit it and forget it.) Seek links from authority sites in your industry. If local search matters to you (more on that coming up), seek links from trusted sites in your geographic area — the Chamber of Commerce, local business directories, etc. Analyze the inbound links to your competitors to find links you can acquire, too. Create great content on a consistent basis and use social media to build awareness and links. (A blog is great for this; see below.)

16. Use press releases wisely. Developing a relationship with media covering your industry or your local region can be a great source of exposure, including getting links from trusted media web sites. Distributing releases online can be an effective link building tactic, and opens the door for exposure in news search sites. Related bonus tip: Only issue a release when you have something newsworthy to report. Don’t waste journalists’ time.

17. Start a blog and participate with other related blogs. Search engines, Google especially, love blogs for the fresh content and highly-structured data. Beyond that, there’s no better way to join the conversations that are already taking place about your industry and/or company. Reading and commenting on other blogs can also increase your exposure and help you acquire new links. Related bonus tip: Put your blog at yourdomain.com/blog so your main domain gets the benefit of any links to your blog posts. If that’s not possible, use blog.yourdomain.com.

18. Use social media marketing wisely. If your business has a visual element, join the appropriate communities on Flickr and post high-quality photos there. If you’re a service-oriented business, use