Transcript: Google & Bing On Whitelists & Exception Lists

Last week at our SMX West show, Google and Bing spoke about how sites might get excepted from being harmed by particular signals in their search algorithms. Whitelists? Exception lists? List overrides? What’s it all mean? Below, a transcript of the answers, so you can read exactly what was said for yourself. The question came […]

Chat with SearchBot

Pic 1299787189Last week at our SMX West show, Google and Bing spoke about how sites might get excepted from being harmed by particular signals in their search algorithms. Whitelists? Exception lists? List overrides? What’s it all mean? Below, a transcript of the answers, so you can read exactly what was said for yourself.

The question came up during the Spam Police session, where Google’s Matt Cutts and Bing’s Sasi Parthasarathy spoke about whether they had whitelists. The session was moderated by Search Engine land editor-in-chief Danny Sullivan.

Sullivan: Matt, do you guys have a whitelist? Sites that, never harm these sites, always let them do well?

Cutts: That is a good question. So when you have an algorithm, you do your very best to make sure it works. There are many, many algorithms in Google. That is like saying, is a car a machine. Well, a car is composed of a bunch of different machines.

So if you take some individual algorithms within Google, you might have something where you will have an exception list, where you say, this algorithm is not perfect, let’s see what we can do until we can fix this algorithm up for the next round.

What I think most people talk about [when they ask about whitelists], “Is there is some type of overriding, you are golden, you can never be touch, either philosophy or list.” And to the best of my knowledge, there is nothing like that.

A query comes in, it doesn’t get touched by people. A bunch of, a hundreds of machines all process it, and it under a couple hundred milliseconds, what you get back is what we think are the best ten results.

Sullivan: So there is no overall, let’s call it the “Wikipedia list,” there is no overall, “This site should always be fine for everything.” But if you have a particular signal that you are implementing and you think, this signal is working well to deal with with 99% of the sites out there but “Wow, it is really hurting this other site for completely unrelated reasons, then let’s exempt them from that” type of thing?

Cutts: Well, I think if you were in charge of Google. Like suppose we all got hit by a bus and you guys had to come into the Googleplex and run the search engine yourselves, right? After you ate all the free food, then the next thing you would do, is you’d think of what is the philosophy, how do we make it work? And I think the right instinct is to try to do as much as you can algorithmically,

It is the same thing as spam. You try to solve hidden text or hacked site algorithmically, and then the stuff you can’t catch you are willing to take manual action on to remove because that is abuse and it is a bad user experience. And then you use that data to try do it the next round, so it is completely algorithmic, or it is algorithmic for more languages.

And I think that is the philosophy like almost anybody would come to, you want to be as scalable as you can.

Sullivan: So that is yes?

Cutts: I think that is a yes.

Sullivan: But going forward, then “We try to improve the algorithm, so that when we update the signal it goes.” And it sounds like Sasi, you have say.

Parthasarathy: I want to say one point to that.

If you want to get technical, this is like an advanced concept. Every algorithm we call what is called a precision and recall. How precisely an algorithm does it job and to what extent did it do its job.

So out of a hundred sites, fifty sites are bad. Is the algorithm able to catch all fifty sites, that is the recall. Just like, go on any search engine and type precision recall evaluation.

None of the algorithms has, as Matt said, have a 100% precision or 100% recall. So that is why we need all these list overrides.

But the interesting part is that any time we have these static overrides, we will make sure that the next iteration of the algorithm actually handles these lists. So these lists are constantly evolving.

It is not like ABC.com or any specificsite.com is always be on the whitelist or always going to be on the blacklist. It just like evolves basically. Because we do have a manual slash algorithmic approach to that.

Cutts: Yea, it is also important to realize that there are many many algorithms, maybe the majority of the algorithms that don’t have any exception logs. For example, Panda, we don’t have, there is not way right now, to do any manual exceptions.

Sullivan: No, no, I read the Cult Of Mac you’d exempted things for them. [sarcasm]

Cutts: Nope.

You can hear this section of the panel in the video below. Sorry, there’s little “video” in the video — only the session title slide was recorded. But the audio is clear.

Related Stories:


About the author

Barry Schwartz
Staff
Barry Schwartz is a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on Twitter here.

Get the must-read newsletter for search marketers.