A reader emailed me today noticing that Google was showing a date next to his listing, which made me think this was a good time to revisit how, when and where search engines show crawl dates for pages. These dates are a useful way for site owners to understand how often they are being revisited or for anyone to "squeeze the loaf" of a search engine to see how fresh it is. Here's a search engine-by-search engine rundown on date display. I'll also cover how we've sadly lost crawl dates being embedded next to listings, over the years. But that's not all! Read now and you'll even [...]

Related Topics: Ask: SEO | Ask: Web Search | Channel: SEO | Google: SEO | Microsoft: Bing | Microsoft: Bing SEO | Search Features: Dates | SEO: Blocking Spiders | SEO: General | SEO: Titles & Descriptions | Stats: Freshness | Yahoo: SEO | Yahoo: Site Explorer

The Google Blog has a basic but clear explanation of the robots exclusion protocol. In this blog post, Google explained how the NOINDEX, NOFOLLOW, NOARCHIVE and NOSNIPPET META tags function. In short: NOINDEX tag tells Google not to index a specific page NOFOLLOW tag tells Google not to follow the links on a specific page NOARCHIVE tag tells Google not to store a cached copy of your page NOSNIPPET tag tells Google not to show a snippet (description) under your Google listing, it will also not show a cached link in the search results [...]

Related Topics: Channel: SEO | Google: SEO | SEO: Blocking Spiders | SEO: General

Since Yahoo rolled out a new Delete URL feature this week, a number of questions have come up on how exactly it works. I had time yesterday with some of the Yahoo Site Explorer team to gather answers. Thanks to Priyank Garg and Amit Kumar, who along with Tim Mayer, went through the inner workings. It's probably most important to understand the difference between how pages have traditionally been kept out of Yahoo versus what Delete URL does. Traditionally, Yahoo is told not to spider pages at all using either a robots.txt file or a meta robots tag that uses the "noindex" setting. [...]

Related Topics: Channel: SEO | SEO: Blocking Spiders | Yahoo: SEO | Yahoo: Site Explorer

The Yahoo Search Blog just announced that Yahoo Site Explorer has new features. Yes, one of those features is the Yahoo Site Explorer Badge, which we reported this morning. But there are more features. You can now delete URLs from the index with a click of a button. If you authenticated your site with Yahoo Site Explorer, there will be a delete button when you "explore" a URL. Other features include authentication via META tags and more detailed authentication errors. Pretty cool that you can easily remove URLs from Yahoo's Search index, here is a screen capture: [...]

Related Topics: Channel: SEO | SEO: Blocking Spiders