From The Isn’t It Ironic Dept: Google Product Search’s Results Show Up In Google

Remember how Google said recently that it might crack down on listings pages that are simply search results themselves? Reader Michael Nguyen dropped an email today to point out how, ironically, Google is now listing pages from its own Google Product Search service exactly as it has warned others not to do.

OK, settle down back there, those of you having a chuckle. Embarrassing? Yes! Intentional? Almost certainly not. Let’s take a look.

Try a search for snake light, and you’ll get this:

Google Product Search Result On Google

See down there at the bottom? Two pages from Google Product Search showing up in the top results:

Google Product Search Result On Google

I can’t resist. Let’s dig the hole for Google just a bit deeper before I throw down a ladder, so it can climb out.

Google’s recent warning about cracking down on search results showing up in its search results IS understandable. I mean, if you’ve done a search for beard trimmer, who wants to get a bunch of pages that just lead you to shopping sites, like this:

Shopping Search Results On Google

See all the results I’ve highlighted in red. Click on any of those, and you end up not at pages giving you information about beard trimmers or a particular beard trimmer product. Instead, you just get shopping search results, pages from shopping search engines listing a variety of beard trimmers and prices from merchants across the web.

For example, click on the number nine listing, and you get:

Google Product Search Results

Oh, yeah, um — shopping results from across the web, courtesy of Google Product Search.

So how did this listing for Google Product Search:

Google Product Search Result On Google

Wind up being listed by Google itself in Google’s ordinary results? Is this a new conspiracy to compel searchers to try Google Product Search?

Nah. After all, Google already uses a product OneBox to push people to Google Product Search whenever it wants. With the beard trimmer search, you can see this at the top of the page:

Google Product Search OneBox

Ah, but people might skip past this, so it’s better to be in the "real" results. This is Google perhaps trying to slip a change like this past us.

Heh. Google doesn’t need to slip that type of thing past anyone. Google’s already started preparing a move for this publicly. Remember, just last week it stopped using OneBox display for news results, putting news into regular results (for more, see here and here. I’ve yet to see this change myself). This is expected to happen to other specialized Google search results, as well.

The explanation is easy. Google almost certainly forgot to block crawling of Google Product Search results by itself and other crawlers.

Look here at the Google Product Search home page:

Google Product Search

See those links below the search box? Those are recent queries people have done (reload the page, and they change to different examples). Click on a link, and they generate search results. Click on them as a search spider, and you’ll index search results — unless you’ve been blocked.

To block those spiders, Google would need the right entries in its robots.txt file. Let’s check!

Oops. No entries. Google does have these:

Disallow: /froogle?
Disallow: /froogle_

Those were in there to stop queries from Google’s Froogle shopping search engine from being indexed. Unfortunately, Google didn’t update these entries to reflect how Froogle was renamed Google Product Search earlier this month.

That renaming shifted product results to this new URL:

As you can see in this URL for [beard trimmer]:

As soon as Google blocks the /products area via the robots.txt file, those 151,000 or so product pages that have been indexed will go away.

I am pinging Google about this, but I know exactly what they’ll say. This was an oversight, and the robots.txt file will be updated soon. So go on, you’ve had your laugh for the night!

Related Topics: Channel: SEO | Google: Google Shopping | Google: OneBox, Plus Box & Direct Answers | Google: SEO | SEO: Blocking Spiders | SEO: Spamming


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • NeO

    That’s Hilarious Danny!!! I bet this makes it’s rounds in the blogosphere before those results are out of the main index.

    Good find.

  • Philipp Lenssen

    They’ve adjusted their robots.txt now to block “/products?”. Those spammers ;)

  • Matt Cutts

    Even though Vanessa Fox did all of the heavy lifting to mention this to the right people, I can’t resist the punchline: Danny, this was an oversight, and the robots.txt file will be updated soon.

    In fact, it already is. Thanks for noticing that in switching from Froogle to Product Search, the robots.txt didn’t add “/products?” to match the “/froogle?”.

  • jeremy mayes
  • bizbuilder

    I just came across the original post, dated 4/30/07. I see some comments say the robots file has been updated to disallow the product searches, yet product searches are still showing up (as of today, 7/7/07). For example, do a Google search on: aluminum liner kit

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide