More Info On Google’s Unavailable After Meta Tag & New X-Robots-Tag In Header Support
Jul 17, 2007 at 11:15am ET by Barry Schwartz
Last week we reported that Google was to add an “Unavailable After” META Tag. Since then, we’ve spoke to Dan Crow of Google, who provided more information on how to use it, as well information on a new way to send robots blocking info within HTTP headers.
The “unavailable_after” Meta tag will allow you to tell Google that a page should expire from the search results at a specific time. For example, if you have a page that you would like to be removed from the search results at 6pm EST on July 23, 2007, you would add the following Meta tag:
<META NAME=”GOOGLEBOT” CONTENT=”unavailable_after: 23-Jul-2007 18:00:00 EST”>
Once Google crawls the page and sees this Meta tag, it will take about a day for the page to be removed from the search results. Also this tag is only supported with web search. To remove the page completely from Google, including the cache copies, use the Google removal tool.
Google has also added support to control access to non-HTML documents that can’t have meta tags in them for blocking, such as PDF files, audio, xls documents and so. This is through a new X-Robots-Tag directive issued via the HTTP Header used to serve the file.
Here are examples of how to use the X-Robots-Tag:
Don’t index this document:
X-Robots-Tag: noindex
Don’t show cache or snippet in search results for this file:
X-Robots-Tag: noarchive, nosnippet
Don’t index after a specific date (i.e. the “unavailable_after” tag):
X-Robots-Tag: unavailable_after: 7 Jul 2007 16:30:00 GMT
You can also combine these tags as well.
Danny has more on meta tags in Meta Robots Tag 101: Blocking Spiders, Cached Pages & More.
Share, Bookmark & Discuss This Article
More:
Keep Updated: News Via Email | News Via RSS Feed | News Via Twitter
See more stories like this in the Members Library! Check out the Google: SEO, SEO: Blocking Spiders sections of the Members Library where this story is filed. Members also get access to exclusive video content, a members-only weekly & monthly newsletter, plus more. Check out all the benefits!
TOP STORIES
SEARCH NEWS BRIEFS
FEATURES & ANALYSIS
RECENT COMMNENTS
- Webmaster T said " If it was a search engine and not the people who actually benefit from proving this relationship... "
- Eric Ward said " OK, so I was trying to be ironic/funny with my maniacal idea for aggregating all my personal comment"
- JohnWEllis said " Greg, Thanks for sharing this data. No good comes from asking people what ads are “helpful”. People "
FREE DAILY SEARCH NEWS RECAP!
Stay on top of all the search news with our daily summary, the SearchCap newsletter. View a sample ›
STAY CURRENT THROUGHOUT THE DAY
Search Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.
SMX Web Site » | SMX Difference » | SMX News »
Join us at an upcoming SMX event:
- SMX Singapore - July 2-3, 2009
- SMX São Paulo - August 4-5
- SMX East - October 5-7, 2009
- SMX Stockholm - 12-13 October, 2009
- SMX Mexico - November 11, 2009
Learn more about search marketing with our free online webcasts and webinars from our sister site, Search Marketing Now. Upcoming online events include:
- How Large Offline Marketers Drive Superior Search Marketing Results - July 21, 2009
- Trademarks, Brand Terms and PPC Advertising: Updates You Must Know - July
22, 2009
- Attribution Management Buyers Guide - July 28, 2009
TRACK US SOCIALLY
- Wall Street Journal Recommended Reading
- Ad Age Power 150 Blog
- Technorati Top 100 Blog
- Techmeme Leaderboard Source
- Best Search Engine Community/Forum
Featured sites from our Blogroll
- evilgreenmonkey
- Search Month: Search News In Review
- Daggle: Danny Sullivan’s Blog
- Bill Hartzer
- Ask the SEO Guru
Get Your Search Engine Land
Premium Membership!
Become a premium member today and receive:
- Express commenting privileges & photo.
- Exclusive videos & newsletters.
- Discounts to our SMX conferences.
- Access to "How To" & Other Archives.
© 2009 Third Door Media, Inc. All rights reserved. Google, Yahoo & Microsoft logos trademark respective companies and used with permission.
