Google Dumps The Supplemental Results Label

Bye Bye Supplemental Results LabelAs many expected, Google has announced that it is dumping the supplemental result label for pages listed within its supplemental index. Does that mean the supplemental index has gone away? No. But Google promises that being in the supplemental index versus the main index should be less of an issue for site owners, going forward. More about the change, below.

Back in September 2003, Google unveiled the supplemental index as a way for it to index more pages from the web but not get bogged down actually searching through those billions and billions of them unless necessary. By and large, it was less important pages that ended up in the supplemental index. If Google’s main index — containing what it considered the best or most important pages – didn’t seem to have a match, then Google would look to the supplemental index for listings.

Pages in the supplemental index could, did and do rank — but only generally if there’s not much to be found in the primary index. As a result, site owners who discovered that many of their pages were in the supplemental index would worry. They interpreted that as a sign that Google saw much of their content to be not as important. And that was a fair enough interpretation. The pages really were "second class" in the sense of being listed only if the main index had nothing gripping plus the fact that Google might not revisit pages in the supplemental index for months.

Concerned site owners have come up with various ways to tell if they have pages in the supplemental index, and one of these methods started failing to work earlier this month. Is Google Gearing Up To Drop The Supplemental Result Label? from us explains what happened more and how a comment from Google’s Matt Cutts prompted speculation that the label would go away.

Matt was indeed laying the groundwork for today’s news. From the Google Webmaster Central blog post, Supplemental goes mainstream:

Since 2006, we’ve completely overhauled the system that crawls and indexes supplemental results. The current system provides deeper and more continuous indexing. Additionally, we are indexing URLs with more parameters and are continuing to place fewer restrictions on the sites we crawl. As a result, Supplemental Results are fresher and more comprehensive than ever. We’re also working towards showing more Supplemental Results by ensuring that every query is able to search the supplemental index, and expect to roll this out over the course of the summer.

The distinction between the main and the supplemental index is therefore continuing to narrow. Given all the progress that we’ve been able to make so far, and thinking ahead to future improvements, we’ve decided to stop labeling these URLs as "Supplemental Results." Of course, you will continue to benefit from Google’s supplemental index being deeper and fresher.

Let’s deal with the labels first. As a reminder, in case you’ve never noticed them, here’s an example of how they’ve looked:

Supplemental Result - Labeled.

That’s my article from April about PageRank. See the words "Supplemental Result" on the third line? That tells you this page is listed in the supplemental index. But that label is going away. Like now. The process has already started, and it will take a few days to rollout completely across various Google data centers, Google told me. If you still see them, it won’t be for long.

At the moment, if you want to force the labels to show up, doing a search for [site:domain/&] is a tip that came out of WebmasterWorld this week, and that still seems to be still working. Search Engine Roundtable has more about it, and we’ve also had discussion on it at Sphinn here. You can see it in action with this search for Search Engine Land:

site:searchengineland.com/&

Ready to be angry and upset that Google is preventing you from knowing a key status indicator of your pages? Hang on, because there’s hope on two different fronts.

First, as the blog post explains, by the end of summer, there should be less difference between the supplemental index and the main one. Google is promising that every query should hit both indexes at the same time, so that a page in supplemental really should have a better chance of ranking than in the past. Google’s also said freshness has increased, so that no page in the supplemental index is more than three months out of date and that this might be reduced to only a month in the future.

Of course, saying freshness has increased to being only "three months old" is kind of a contradiction. It’s true that many pages in the supplemental index won’t change very often. But some do, so it would be good for Google to step up an event faster crawl of supplementals and kick more of them out into the main index, as warranted. The balancing act, of course, is you can have site owners making very minor changes to poor quality pages in hopes of getting out of the supplemental index, so a page change alone can’t be the sole criteria for moving a URL out of the supplemental index.

Promises aside, site owners should be able to know if they have page in the supplemental index. Ideally, I’d like to see a tool within Google Webmaster Central that would provide a list of these, or a percentage of pages from a site that are deemed supplemental as a health check, or both.

In talking with Matt, he expressed concern that this type of tool might cause site owners to needlessly fixate too much on supplementals as many have already been doing and similar to how many have obsessed for years over PageRank. Still, he said that Google would probably come up with a way for people to perform a query within Google Webmaster Central or some other method to find out if a page or pages are in the supplemental index.

Let’s hope that happens and soon. If the labels are going, an alternative method should be brought in as fast as possible. And despite concerns that site owners might obsess, I’d still like some nice reporting. After all, Google Webmaster Central already tells site owners their PageRank distribution. Reporting metrics on supplemental indexing makes as much sense.

Finally, remember that example above with the URL showing a supplemental label? Here it is for a regular search, for pagerank, appearing without the label:

Supplemental Results -- Without Label

Right now, the page ranks around 60 — pretty bad if you’re looking for traffic, but pretty good for a fairly new article on a very popular subject. In addition, doing some testing, I can tell that it is outranking some pages that are in the primary index. So being in the supplemental index clearly isn’t the kiss of death — but you’d still like to know the status. Providing that would help Google better prove its point. If people can easily tell that a supplemental page is outranking a page in the primary index, then concerns will actually ease. Just dumping the label alone won’t make those concerns go away.

Postscript: Google sent along this tip for finding supplemental pages now that the label is going, which I think will be part of what Matt will be posting on his own blog later.

First, get a list of all of your pages. Next, go to the webmaster console [Google Webmaster Central] and export a list of all of your links. Make sure that you get both external and internal links, and concatenate the files.

Now, compare your list of all your pages with your list of internal+external backlinks. If you know a page exists, but you don’t see that page in the list of site with backlinks, that deserves investigation. Pages with very few backlinks (either from other sites or internally) are also worth checking out.

I’d still rather go to to Google Webmaster Central, select the "export supplemental URLs" option and get a list that way!

Related Topics: Channel: SEO | Google: SEO

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://www.seo4fun.com/blog/ Halfdeck

    “So being in the supplemental index clearly isn’t the kiss of death”

    How do you draw a conclusion based on one URL? Not all supplemental URL ranks 0-60 for competitive terms. In fact, I don’t remember seeing any supplemental result for terms like “car insurance” in the first 300 results.

  • http://searchengineland.com Danny Sullivan

    And you’ll draw a conclusion from that one result?

    I said in the article already that in the past, you’d only be seeing results for supplemental listings coming up after primary listings were consulted. I think I was pretty clear on that, in terms of how sucky it would be.

    With the change, yes, I’m pointing at one example where I can see a supplemental listing showing up ahead of primary listings, which does indeed show that supplemental listings aren’t necessarily never to be competitive, as some might assume.

    Still — and again I think I’m pretty clear on this — I’d rather Google made it easier for anyone to check and know if a supplemental result is ranking competitively against regular ones, as they say will be the case. Then we don’t have to poke and guess, and a bunch of people can share experiences to determine how bad — or not bad — being in supplemental will be.

  • http://www.seo4fun.com/blog/ Halfdeck

    “And you’ll draw a conclusion from that one result?”

    Danny, where did I draw one?

    “I’m pointing at one example where I can see a supplemental listing showing up ahead of primary listings, which does indeed show that supplemental listings aren’t necessarily never to be competitive, as some might assume.”

    Okay, then I wish you wrote “so being in the supplemental index isn’t necessarily the kiss of death.”

    If a robots.txt disallowed URL shows up as a supplemental result, for example, that’s nothing to worry about. And like you said, supplemental URLs sometimes outrank URLs in the main index.

    But I don’t buy your specific example because some URLs show up as both supplemental and non-supplemental, depending on the query. In the case of the SEL article ranking for “PageRank”, it isn’t clear if the URL ranking is really supplemental. Notice your second screenshot doesn’t show a “supplemental results” label.

  • http://www.searchengineworkshops.com RNobles

    Danny,

    Phenomenal article and extremely insightful. Congratulations to Google for taking steps in the right direction. As you so aptly stated, more needs to come, but at least they’re headed in the right direction.

    The SI has been seen as a nightmare to so many people for so long. I sincerely hope things change in more ways than just the labels.

    Robin

  • http://www.seo-theory.com/wordpress/ Michael Martinez

    Being in the Supplemental Index HAS actually been “the kiss of death” since at least a year ago. Google has now openly admitted on more than one occasion (including this latest announcement) that pages in the Supplemental Index have not been fully parsed and indexed.

    Even now we are being told that we’ll have to wait a little longer for Supplemental Results pages to appear in ALL queries (because they were not appearing in MOST queries until recently).

    I have already noticed (and reported on improvements in Google’s handling of Supplemental Pages). I’m encouraged because Supplemental Pages NOW show up in queries.

    But Google needs to be fair about this. Just because a page doesn’t have many inbound links is no reason to prevent it from having an equal opportunity to score well for relevance. We already know that spammers boost their linkage and that many SEOs rely extensively on irrelevant lnkage to make Web pages look relevant to the algorithm.

    The real issue has always been that Google allows pages to pass link anchor text. When they stop playing that silly game, the problem will be completely resolved for them.

  • http://searchengineland.com Danny Sullivan

    @Halfdeck, agreed — “so being in the supplemental index isn’t necessarily the kiss of death.” That’s how I meant it :)

    But I don’t buy your specific example because some URLs show up as both supplemental and non-supplemental, depending on the query.

    My understanding was that a URL is either in the supplemental or not, not in both places. If that’s correct, then…

    “Notice your second screenshot doesn’t show a ‘supplemental results’ label.” is correct, since I was doing it after the announcement came out, when Google said the labels would be going away, and not using a command that make that label trigger on.

    If I’m incorrect, point taken!

  • http://www.seo-theory.com/wordpress/ Michael Martinez

    Danny: “My understanding was that a URL is either in the supplemental or not, not in both places. If that’s correct, then…”

    Different crawlers, different indexes. A page can be in both at the same time. I’ve watched Google snatch pages for Supplemental and then for Main Web.

    Whether they have a system in place to remove pages from the Supplemental Index after they hit the Main Web Index is not something I can determine — and they haven’t said anything on the topic that I’m aware of.

  • http://www.igorthetroll.com Igor The Troll

    Danny, what did Matt recommend? Export Dogs and Import cats?

    That’s against Google policy, because they clearly state in the Googler rules of conduct that Google Inc. is a dog company. :)

    Okay jokes aside and a bit Thumbs up to poor old Matt, Google supplementary are more fresh and relevant than before. And thanks for finally getting rid of the persecution of Websites!!!

    A search for a medium tail keyword phrase gets great hits on the SUPs

    So Google is doing its thing…I just learned from Vanessa that she is slow but sure. :)

    Looks like Google is too…

    I actually got that phrase from my nice Thai friend Am…we always joke, “Slow but sure!”

    But when there are no results, it goes, “Slow and not sure!” Sort of American joke as we like to say in Japan, if it goes over someone’s head…

    Thanks for the tip of how to get the SUP’s from the WMF people…

    I found some people saying that the query does not work because it brings strange results…

    Well I checked it against my method that shows results of exSUP’s and its right on the money…

    So searchengineland.com has 3,676 main + 304 SUP’s + 14 skeleton URL’s =3994 total
    the site: 3,980

    Danny boy, what are you hiding in your closet?
    :)

    Okay, the Skeleton urls are the ones that are left over after you do a URL removal request and someone still links to it, probably some proxy sites…

    It would be nice if Mr. C can suppress them as he did the SUP tag…we do not want to show our skeletons Matt.

    Okay so this was my little contribution to the Google melting pot…I am of to China on Tuesday…and will give it a break from the social networking…I hope Adam does not miss me. :)

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide