Everything you need to know about SEO, delivered every Thursday.
Google Dumps The Supplemental Results Label
As many expected,
announced that it is dumping the supplemental result label for pages listed
within its supplemental index. Does that mean the supplemental index has gone
away? No. But Google promises that being in the supplemental index versus the
main index should be less of an issue for site owners, going forward. More about
the change, below.
Back in September 2003, Google unveiled the supplemental index as a way for
it to index more pages from the web but not get bogged down actually searching
through those billions and billions of them unless necessary. By and large, it
was less important pages that ended up in the supplemental index. If Google’s
main index — containing what it considered the best or most important pages —
didn’t seem to have a match, then Google would look to the supplemental index
Pages in the supplemental index could, did and do rank — but only generally
if there’s not much to be found in the primary index. As a result, site owners
who discovered that many of their pages were in the supplemental index would
worry. They interpreted that as a sign that Google saw much of their content to
be not as important. And that was a fair enough interpretation. The pages really
were "second class" in the sense of being listed only if the main index had
nothing gripping plus the fact that Google might not revisit pages in the
supplemental index for months.
Concerned site owners have come up with various ways to tell if they have
pages in the supplemental index, and one of these methods started failing to
work earlier this month.
Is Google Gearing Up To Drop The Supplemental Result Label? from us explains
what happened more and how a comment from Google’s Matt Cutts prompted
speculation that the label would go away.
Matt was indeed laying the groundwork for today’s news. From the Google
Webmaster Central blog post,
Supplemental goes mainstream:
Since 2006, we’ve completely overhauled the system that crawls and indexes
supplemental results. The current system provides deeper and more continuous
indexing. Additionally, we are indexing URLs with more parameters and are
continuing to place fewer restrictions on the sites we crawl. As a result,
Supplemental Results are fresher and more comprehensive than ever. We’re also
working towards showing more Supplemental Results by ensuring that every query
is able to search the supplemental index, and expect to roll this out over the
course of the summer.
The distinction between the main and the supplemental index is therefore
continuing to narrow. Given all the progress that we’ve been able to make so
far, and thinking ahead to future improvements, we’ve decided to stop labeling
these URLs as "Supplemental Results." Of course, you will continue to benefit
from Google’s supplemental index being deeper and fresher.
Let’s deal with the labels first. As a reminder, in case you’ve never noticed
them, here’s an example of how they’ve looked:
That’s my article
from April about PageRank. See the words "Supplemental Result" on the third line? That tells you this
page is listed in the supplemental index. But that label is going away. Like
now. The process has already started, and it will take a few days to rollout
completely across various Google data centers, Google told me. If you still see
them, it won’t be for long.
At the moment, if you want to force the labels to show up, doing a search
for [site:domain/&] is a tip that
came out of
WebmasterWorld this week, and that still seems to be still working.
Roundtable has more about it, and we’ve also had discussion on it at Sphinn
here. You can see it in action with this
search for Search Engine Land:
Ready to be angry and upset that Google is preventing you from knowing a key
status indicator of your pages? Hang on, because there’s hope on two different
First, as the blog post explains, by the end of summer, there should be less
difference between the supplemental index and the main one. Google is promising
that every query should hit both indexes at the same time, so that a page in
supplemental really should have a better chance of ranking than in the past.
Google’s also said freshness has increased, so that no page in the supplemental
index is more than three months out of date and that this might be reduced to
only a month in the future.
Of course, saying freshness has increased to being only "three months old" is
kind of a contradiction. It’s true that many pages in the supplemental index
won’t change very often. But some do, so it would be good for Google to step up
an event faster crawl of supplementals and kick more of them out into the main index, as
warranted. The balancing act, of course, is you can have site owners making very
minor changes to poor quality pages in hopes of getting out of the supplemental
index, so a page change alone can’t be the sole criteria for moving a URL out of
the supplemental index.
Promises aside, site owners should be able to know if they have page
in the supplemental index. Ideally, I’d like to see a tool within Google Webmaster Central that would provide a list of these, or a percentage of pages
from a site that are deemed supplemental as a health check, or both.
In talking with Matt, he expressed concern that this type of tool might cause
site owners to needlessly fixate too much on supplementals as many have already
been doing and similar to how many have obsessed for years over PageRank. Still, he
said that Google would probably come up with a way for people to perform a query
within Google Webmaster Central
or some other method to find out if a page or pages are in the supplemental
Let’s hope that happens and soon. If the labels are going, an alternative
method should be brought in as fast as possible. And despite concerns that
site owners might obsess, I’d still like some nice reporting. After all, Google
Webmaster Central already tells site owners their PageRank distribution.
Reporting metrics on supplemental indexing makes as much sense.
Finally, remember that example above with the URL showing a supplemental
label? Here it is for a regular search, for
Right now, the page ranks around 60 — pretty bad if you’re looking for traffic,
but pretty good for a fairly new article on a very popular subject. In addition,
doing some testing, I can tell that it is outranking some pages that are in the
primary index. So being in the supplemental index clearly isn’t the kiss of
death — but you’d still like to know the status. Providing that would help Google better prove its point. If people can easily tell that a
supplemental page is outranking a page in the primary index, then concerns will
actually ease. Just dumping the label alone won’t make those concerns go away.
Postscript: Google sent along this tip for finding supplemental pages
now that the label is going, which I think will be part of what Matt will be
posting on his own blog later.
First, get a list of all of your pages. Next, go to the webmaster console
[Google Webmaster Central] and export a list of all of your links. Make sure
that you get both external and internal links, and concatenate the files.
Now, compare your list of all your pages with your list of
internal+external backlinks. If you know a page exists, but you don’t see that
page in the list of site with backlinks, that deserves investigation. Pages
with very few backlinks (either from other sites or internally) are also worth
I’d still rather go to to Google Webmaster Central, select the "export
supplemental URLs" option and get a list that way!