Google, Content Farms & Why This May Be Blekko’s Moment

At the beginning of Google’s “Searchology” event in early 2007 original Google employee Craig Silverstein opined, “If Google had started a year or two earlier, it wouldn’t have worked.” That’s because prior to that time (1998) the internet wasn’t yet large enough to require Google or enable people to see the value of its approach.

As sites and pages multiplied exponentially Google became an increasingly necessary tool. We’re all familiar with the story. Now Google controls a majority of search traffic in most countries around the world. It has become a seemingly unstoppable force.

The influence of search (paid and organic) has been so powerful that billions of dollars have changed hands and established media companies have been all-but-toppled by their failures to recognize and exploit search effectively. Newspapers in particular fall into the latter category.

But the lessons of search and SEO have been well-learned by some media companies, many entrepreneurs and investors. In response, a range of so-called “content farms” has arisen to drive page views off “content” created by hundreds of mostly low-paid bloggers (and some former journalists).

Demand Media, Associated Content (now part of Yahoo), Examiner.com, Suite 101 and others recruit and train freelancers to quickly generate articles on all manner of niche topics that will drive qualified page views or lead generation in some cases. Yahoo just made a $100 million bet on this version of SEO. And AOL is pursuing a somewhat more refined version of this same strategy with Seed.

The original model arguably is “The Mining Company,” which in 1999 became About.com and was later acquired by the NY Times for its SEO/link value. While About.com was originally about “content curation” (organizing and commenting on links), the content farms are about content creation.

Despite the protests of the companies themselves about the terms “content farm” and “content mill,” the truth is that the articles and columns produced by these operations are of variable quality at best. And in some cases the content they generate should be considered a form of spam.

The proliferation of dubious or low-quality content from some of these sites is, over time, a direct threat to Google in my view. Google doesn’t present any publisher “branding” in search results so users must often click back and forth to find a quality source for the information they’re seeking. Google’s algorithm is supposed to address issues of quality and authority — and it often does — but the rising tide of mediocre, SEO-driven content creation is a fundamental problem for searchers.

Enter Blekko.

Over the past several years there have been many runs at Google and general search, including Powerset (acquired by Microsoft) and the ill-fated Cuil. None of these independent challengers has succeeded to date. (The jury’s still out on Bing of course, but arguably it has already succeeded by several measures.)

For those who haven’t heard of it, Blekko is a general search engine that will launch relatively soon. It has been written about several times by TechCrunch and I wrote about it briefly a year ago after an initial demo from founder Rich Skrenta and Mike Markson. Danny intends to do a “deep dive” on Blekko shortly so I’ll let him discuss features in depth.

There are two general characteristics that differentiate Blekko: transparency and user control. It’s also social in interesting ways; registered users can “follow” one another.

Blekko’s motto is “Slash the Web.” The centerpiece of that slogan and its chief innovation is the concept of “slashtags.”

Slashtags allow search personalization and filtering through the creation of mini-indexes of authoritative or favorite sites. For example, Skrenta has created a slashtag for wine blogs that he likes: “/skrenta/wineblogs.” In short this allows him or me to search an authoritative or personal sub-index of the internet for wine-related content and recommendations.

This way I can get articles and commentary from sites I trust or that people I trust recognize as authoritative — and cut out the, pardon the expression, crap content.

While there’s a short learning curve slashtags are easy to use and their value is almost self-evident. Blekko can also be used just like Google without slashtags as well.

Slashtags address the content-spam problem I described and provide control over results not offered by Google today. My prediction is that sophisticated search users will immediately be drawn to Blekko for the personalization, SEO tools (I’ll let Danny discuss) and social features it offers. I also predict that after it formally launches we may see some slashtag-like development from Google.

In 1998 the web was ripe for Google. But Google’s profound success and the way it has shaped the internet, giving birth to content farms, may have paved the way in 2010 for Blekko.

NOTE: Please see our detailed review that’s now available, Blekko: New Search Engine Lets You “Spin” The Web.