Yahoo Provides NOYDIR Opt-Out Of Yahoo Directory Titles & Descriptions

Danny Sullivan on February 28, 2007 at 2:06 pm

Yahoo! Search
Support for ‘NOYDIR’ Meta Tags and Weather Update from the Yahoo Search Blog
covers how at long last, you can now tell Yahoo to not use Yahoo Directory
information to make a title and/or description for your web page listings. It
also cover how Yahoo’s currently doing a reindexing change that might impact
rankings. More on that below, plus tips about also blocking the Open Directory
information from being used for your pages and some possible conflicts with
multiple robots tags.

Sometimes pages are listed in both Yahoo’s crawler-based search results and
within its human-compiled directory, the Yahoo
Directory. In those cases, Yahoo usually replaces the title and description
of a web page in the crawler-based results with the information from the Yahoo
Directory. Yahoo has operated this way for as long as I can remember — that’s
over a decade :)

Now this has changed. Sometimes a site owner might not want the Yahoo
Directory description to be used for their page. A case in point is Tony
Knowles. The Yahoo Directory
lists him this way:

Knowles, Tony (D)
Democratic candidate from Alaska for U.S. Senate, 2004.
www.tonyknowles.com

So when you search for him in Yahoo web search service like this —
tony knowles — his
listing comes up as so:

Tony Knowles
Democratic candidate from Alaska for U.S. Senate, 2004
Category:

Alaska > 2004 U.S. Senate Election
www.tonyknowles.com – 15k –

Cached –

More from this site

The problem is, Yahoo’s directory information outdated. Knowles did run for
Senate in 2004, but then he ran for governor of Alaska in 2006 using the same
web site. During his governor campaign, this caused Yahoo to list him as a
senate candidate, as I
covered last
year.

Now there’s a solution. By inserting this meta tag on any page:

<META NAME="ROBOTS" CONTENT="NOYDIR">

You can ensure that Yahoo will not use a directory description it might have
for that page.

<META NAME="Slurp" CONTENT="NOYDIR">

The first tag tells any spider that wants to recognize the tag not to use a
Yahoo Directory title or description. Of course, no other spiders do that — but
Yahoo’s just building in some protection should that come up in the future. The
second tag specifically tells the Yahoo spider not to use the information.

A related tag is the NOODP meta tag. Similar to how Yahoo works, various
search engines have looked at the Open Directory
Project to get information to make titles and descriptions of pages they
also crawl. Pressure started
building
back in 2005 that site owners should be able to prevent Open Directory
information from being used on their pages in this way. Last year, we saw all
the major search engines do it except Ask. Instructions and dates of
implementation for each are below:

That solved the ODP issue, but
pressure remained
on Yahoo to provide a fix for its own directory. Now that’s happened.

Back to the ODP. The tags to use to block the ODP are similar. Here’s the one
to use for all spiders:

<META NAME="ROBOTS" CONTENT="NOODP">

And if you wanted to block both the Open Directory and Yahoo Directory titles
being used, you’ll need to do both of there:

<META NAME="ROBOTS" CONTENT="NOYDIR">
<META NAME="ROBOTS" CONTENT="NOODP">

Maybe. Then again, having two meta robots tags possibly might make a search
engine choose one or the other, not both. You could be safe by using a tag for
each specific spider:

<META NAME="GOOGLEBOT" CONTENT="NOODP">
<META NAME="SLURP" CONTENT="NOODP">
<META NAME="MSNBOT" CONTENT="NOODP">

But that’s a lot of unnecessary work, likely. Instead, I suspect that you
really need to do something like this:

<META NAME="ROBOTS" CONTENT="NOYDIR,NOODP">

That’s consistent with the long-standing standard for the meta robots tag in
general, as covered here.
And if you’re already using the meta robots tag for other things, such as to
block archiving (here are

instructions from Google), you probably have to do this:

<META NAME="ROBOTS" CONTENT="NOYDIR,NOODP,NOARCHIVE">

By the way, upper or lower case — it doesn’t matter. I’m also fairly sure
that putting spaces after the commas like below works:

<META NAME="ROBOTS" CONTENT="NOYDIR, NOODP, NOARCHIVE">

I’ll do some pinging to get answers to some of the questions above. While
it’s great everyone has rolled out support to extend the meta robots tag,
they’ve unfortunately not come together to answer clearly some of the questions
I’ve raised above.

Postscript: I’ve asked Google, Microsoft and Yahoo the same three
questions:

Will there be a problem with two robots tags? Will you go with one or the
other (and if so, which one)
Will you recognize a tag like this:
<META NAME="ROBOTS" CONTENT="NOYDIR,NOODP,NOARCHIVE">
Are spaces also OK like this?
<META NAME="ROBOTS" CONTENT="NOYDIR, NOODP, NOARCHIVE">

Postscript, Answers from Yahoo:

Didn’t answer
They do recognize multiple attributes/commands in the same tag, but they
didn’t explicitly say if this works if there are no spaces after commas for
each attribute (it is probably fine)
Yes, they definitely could handle a tag with multiple attributes and
spaces

About The Author

Danny Sullivan

Danny Sullivan is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.