• kching

    maybe I haven’t had enough coffee :)
    but don’t you mean in you’re second example?
    meta name=”robots” content=”index,follow

  • http://searchengineland.com Danny Sullivan

    Thanks for the catch! Got it fixed.

  • pratt

    I completely agree that it is silly to put:
    meta name=”robots” content=”index,follow” in your code. But if it is already in there, are there any benefits to going in and taking it out? Will it speed up load times or anything like that?

    Please excuse my lack of programming knowledge.

  • http://searchengineland.com Danny Sullivan

    You can safely leave it in :)

  • http://www.antezeta.com/avoid-search-engine-indexing.html Sean Carlos

    As always, a very informative resource.

    Just a quick comment to note that Microsoft does support “noarchive”, so the Microsoft only “nocache” is not needed.

    Reference:

    http://search.live.com/docs/siteowner.aspx?t=SEARCH_WEBMASTER_REF_RestrictAccessToSite.htm

    - Sean Carlos

  • http://searchengineland.com Danny Sullivan

    Thanks, Sean, much appreciated and have it updated.

  • Robert_Charlton

    How does the noindex command on AdWords landing pages affect AdsBot and quality score spidering? Google indicates that it will ignore robots.txt so it can spider these pages, but will it spider them if you’re using the robots meta tag?

    There are many reasons, I feel, why using the robots meta to block indexing of a landing page might be preferable.

  • Robert_Charlton

    Danny – Another comment… related to my comment above, and possibly something to mention if you talk to responsible people at MSN.

    In the past several days, a client site has been plagued by MSN indexing (and prominently ranking) url-only listings to “blocked” pages. These pages have all used the robots meta tag, with the following syntax…

    They include AdWords landing pages, test pages, etc. In the past, I’ve encountered problems with Google indexing such references, but that is because the pages were blocked by robots.txt and the “link references” to these pages were exposed. That’s why I ultimately switched to using the robots meta tag and dropped the robots.txt.

    As we’ve discussed at SEWF, all the engines need to get on the same page (so to speak) about this.

  • http://www.gadgetduck.com/ Igor

    Microsoft’s Livesearch is ignoring the NOINDEX tag on my pages. I have been e-mailing back and forth with them for a few weeks and things just get curiouser and cruriouser. If you do this search for pages at my site -
    http://search.msn.com/results.aspx?q=www.gadgetduck.com&form=QBRE
    - you’ll see some pages with a link labeled “cached page”. Those pages carry the NOINDEX metatag. (Also, those pages carry NOARCHIVE. Livesearch actually does not archive the pages – the “cached page” link comes up blank, which is fine with me. But, this seems to show another system glitch for them.)

    Has anyone else seen this problem? Is it possible that my tags are malformed? Google and Yahoo seem to have no problem following them correctly.

  • maxitudo

    hello, you can give a look at the site and give me your opinion? wwww.showdalotofacil.com grateful moraes