Sign Up To Receive This Content Via Email:  

Google Warning Against Letting Your Search Results Get Indexed

The days of doing a Google search that brings up results leading to search results from other sites are heading for a close. Matt Cutts, in his Search Results In Search Results post today, points out a change to Google's guidelines that shows a crackdown on this type of material may begin. More about what I'm talking about below, plus the question of whether Google should do the same with paid listings. Over time, more and more pages seem to show up in Google search results that are merely lists of search results from those sites. To illustrate this, consider a search for dvd playe [...]

Meta Robots Tag 101: Blocking Spiders, Cached Pages & More

Last week, I covered a new command for the meta robots tag -- one to prevent search engines from using Yahoo titles and descriptions. In doing that, a number of questions came up about the meta robots tag syntax itself. Google Webmaster Central has now posted "Using the robots meta tag," providing some clarity from Google. In addition, both Yahoo and Microsoft have also sent me information on using the tag. I'll run through what everyone says below, complete with charts for easy at-a-glance comparisons. The meta robots tag was an open standard created over a decade ago and designed initiall [...]

Yahoo Provides NOYDIR Opt-Out Of Yahoo Directory Titles & Descriptions

Yahoo! Search Support for 'NOYDIR' Meta Tags and Weather Update from the Yahoo Search Blog covers how at long last, you can now tell Yahoo to not use Yahoo Directory information to make a title and/or description for your web page listings. It also cover how Yahoo's currently doing a reindexing change that might impact rankings. More on that below, plus tips about also blocking the Open Directory information from being used for your pages and some possible conflicts with multiple robots tags. Sometimes pages are listed in both Yahoo's crawler-based search results and within its human-compil [...]

Finding Search Engine Freshness & Crawl Dates

A reader emailed me today noticing that Google was showing a date next to his listing, which made me think this was a good time to revisit how, when and where search engines show crawl dates for pages. These dates are a useful way for site owners to understand how often they are being revisited or for anyone to "squeeze the loaf" of a search engine to see how fresh it is. Here's a search engine-by-search engine rundown on date display. I'll also cover how we've sadly lost crawl dates being embedded next to listings, over the years. But that's not all! Read now and you'll even [...]


The Google Blog has a basic but clear explanation of the robots exclusion protocol. In this blog post, Google explained how the NOINDEX, NOFOLLOW, NOARCHIVE and NOSNIPPET META tags function. In short: NOINDEX tag tells Google not to index a specific page NOFOLLOW tag tells Google not to follow the links on a specific page NOARCHIVE tag tells Google not to store a cached copy of your page NOSNIPPET tag tells Google not to show a snippet (description) under your Google listing, it will also not show a cached link in the search results [...]

Up Close With Yahoo’s New Delete URL Feature

Since Yahoo rolled out a new Delete URL feature this week, a number of questions have come up on how exactly it works. I had time yesterday with some of the Yahoo Site Explorer team to gather answers. Thanks to Priyank Garg and Amit Kumar, who along with Tim Mayer, went through the inner workings. It's probably most important to understand the difference between how pages have traditionally been kept out of Yahoo versus what Delete URL does. Traditionally, Yahoo is told not to spider pages at all using either a robots.txt file or a meta robots tag that uses the "noindex" setting. [...]

Yahoo Site Explorer Adds Features: Delete URL From Index

The Yahoo Search Blog just announced that Yahoo Site Explorer has new features. Yes, one of those features is the Yahoo Site Explorer Badge, which we reported this morning. But there are more features. You can now delete URLs from the index with a click of a button. If you authenticated your site with Yahoo Site Explorer, there will be a delete button when you "explore" a URL. Other features include authentication via META tags and more detailed authentication errors. Pretty cool that you can easily remove URLs from Yahoo's Search index, here is a screen capture: [...]

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide