Oh Good Grep! Web Grepper: A New Web Intelligence Feature From Blekko

Blekko, the search engine that launched last year and is known for its spam reduction efforts, is introducing a new feature today that will help make previously difficult, if not impossible to find (for free), web data accessible to more searchers, marketers, media, and others.

Say hello to Web Grepper. More about the name in a moment.

What’s a Web Grepper?

Web Grepper allows users to run searches on the ENTIRE Blekko index (more than 20 billion URLs according to a recent presentation) to reveal data that you could not search and find using keywords. In other words, you can also search material in the actual HTML coding of a web page.

It’s another tool for your web intelligence toolbox.

Here are a few examples of Web Greps that Blekko has run during the testing:

1. The number of sites utilizing Facebook Comments vs. sites using Disqus

2. The number of urls and domains using nofollow tags

3. Sites with rickroll links

4. Determining which type of feed is more popular, ATOM or RSS? (see image below; click for larger version)

Given the increasing amount of data included in the coding of a web page these days the possibilities are many. Plus, having this data and being able to see trends over time will also be both interesting and helpful.

How Can You Use Web Grepper

Simply head to the Blekko homepage and enter the slashtag /webgrep into the search box.  Next, click the request a report button and enter what you would like to have grepped. Next
, Blekko’s community of  users vote on what greps to run. Blekko will run reports twice a day, Monday-Friday.

Blekko also said they’ll be reviewing reports to make sure that Web Grepper is not be used as a hacking tool to obtain personal and private info.

The Name Web Grepper

Web Grepper is based on a command-line text matching search tool originally developed for UNIX named grep.  For more background here’s the Wikipedia entry.

Disclosure: I have a small amount of Blekko stock options that I received after doing some consulting for Blekko in 2009. I have not consulted for the company since that time.