As sites and pages multiplied exponentially Google became an increasingly necessary tool. We’re all familiar with the story. Now Google controls a majority of search traffic in most countries around the world. It has become a seemingly unstoppable force.

The influence of search (paid and organic) has been so powerful that billions of dollars have changed hands and established media companies have been all-but-toppled by their failures to recognize and exploit search effectively. Newspapers in particular fall into the latter category.

But the lessons of search and SEO have been well-learned by some media companies, many entrepreneurs and investors. In response, a range of so-called “content farms” has arisen to drive page views off “content” created by hundreds of mostly low-paid bloggers (and some former journalists).

Demand Media, Associated Content (now part of Yahoo), Examiner.com, Suite 101 and others recruit and train freelancers to quickly generate articles on all manner of niche topics that will drive qualified page views or lead generation in some cases. Yahoo just made a $100 million bet on this version of SEO. And AOL is pursuing a somewhat more refined version of this same strategy with Seed.

The original model arguably is “The Mining Company,” which in 1999 became About.com and was later acquired by the NY Times for its SEO/link value. While About.com was originally about “content curation” (organizing and commenting on links), the content farms are about content creation.

Despite the protests of the companies themselves about the terms “content farm” and “content mill,” the truth is that the articles and columns produced by these operations are of variable quality at best. And in some cases the content they generate should be considered a form of spam.

The proliferation of dubious or low-quality content from some of these sites is, over time, a direct threat to Google in my view. Google doesn’t present any publisher “branding” in search results so users must often click back and forth to find a quality source for the information they’re seeking. Google’s algorithm is supposed to address issues of quality and authority — and it often does — but the rising tide of mediocre, SEO-driven content creation is a fundamental problem for searchers.

Enter Blekko.

Over the past several years there have been many runs at Google and general search, including Powerset (acquired by Microsoft) and the ill-fated Cuil. None of these independent challengers has succeeded to date. (The jury’s still out on Bing of course, but arguably it has already succeeded by several measures.)

For those who haven’t heard of it, Blekko is a general search engine that will launch relatively soon. It has been written about several times by TechCrunch and I wrote about it briefly a year ago after an initial demo from founder Rich Skrenta and Mike Markson. Danny intends to do a “deep dive” on Blekko shortly so I’ll let him discuss features in depth.

There are two general characteristics that differentiate Blekko: transparency and user control. It’s also social in interesting ways; registered users can “follow” one another.

Blekko’s motto is “Slash the Web.” The centerpiece of that slogan and its chief innovation is the concept of “slashtags.”

Slashtags allow search personalization and filtering through the creation of mini-indexes of authoritative or favorite sites. For example, Skrenta has created a slashtag for wine blogs that he likes: “/skrenta/wineblogs.” In short this allows him or me to search an authoritative or personal sub-index of the internet for wine-related content and recommendations.

This way I can get articles and commentary from sites I trust or that people I trust recognize as authoritative — and cut out the, pardon the expression, crap content.

While there’s a short learning curve slashtags are easy to use and their value is almost self-evident. Blekko can also be used just like Google without slashtags as well.

Slashtags address the content-spam problem I described and provide control over results not offered by Google today. My prediction is that sophisticated search users will immediately be drawn to Blekko for the personalization, SEO tools (I’ll let Danny discuss) and social features it offers. I also predict that after it formally launches we may see some slashtag-like development from Google.

In 1998 the web was ripe for Google. But Google’s profound success and the way it has shaped the internet, giving birth to content farms, may have paved the way in 2010 for Blekko.

NOTE: Please see our detailed review that’s now available, Blekko: New Search Engine Lets You “Spin” The Web.

DMOZ boosts tens of thousands of volunteer editors and millions of websites in more than a half-million categories and content in more than 80 languages. Happy Birthday DMOZ!

Yahoo, a few months ago, celebrated their 14th birthday, with the launch of their directory on March 2, 1995.

Everybody loves Google, everybody loves Wikipedia – so why doesn’t everybody love DMOZ?

Ouch! I mean, I kind of agree, but in some ways don’t. Let’s not forget, Yahoo was one of the first true web directories. Back in the day, everyone wanted to be included in Yahoo. DMOZ/ODP also was very sought after in the early days. People used web directories as search engines. As Google became more popular, people slowly stopped using web directories and opted for search engines. Then in 2004, Google dropped the directories tab from their default menu, which hurt the ODP more. In fact, back then, in my 2004 post, I showed how web directories have their purpose.

The DMOZ blog post goes through examples of how DMOZ does a better job at providing quality results over Google. Of course, anyone can show faults in Google or even DMOZ and claim victory in specific cases. But web directories do have their purposes.

So let’s give DMOZ some respect, shall we? Maybe not? As many webmasters and SEOs know, getting listed in the ODP directory can be challenging to say the least. In the old days, I personally was an editor and I can tell you, I stunk at the job. I rarely logged in to review submissions and then even more rarely approved any sites. A recent SEOmoz post named Want to Get Listed in DMOZ? Become an Editor shows how DMOZ listings can be somewhat biased and even corrupt. Of course, not all of the directory is managed this way, but the directory is huge and these things happen more often then not.

Just the other day, we received an email from a web directory who was upset they get no respect either. Philip, the owner of a directory sent us an email that starts off reading:

Whenever any SEO site talks about web directories it mentions Joe Ant, Massive Links, GoGuides, Rubberstamped, Aviva etc. For five years we’ve been building what we think will eventually be the world’s best directory at http://www.findouter.com and yet no one talks about us.

He asked us to compare a US Hospitals listing at http://www.findouter.com/NorthAmerica/USA/Health/Hospitals versus http://www.dmoz.org/Health/Medicine/Facilities/Hospitals/North_America/United_States/. Okay, so DMOZ has about 40% less listed in their directory under that category. I did not go through each listing to see if each URL is still active and returns a quality result.

I am not sure if the directories warrant more attention from us, but I would agree that on some level, they do deserve our respect. But I do wonder, would we have a search engine today if it wasn’t for the early web directories?

• Provide authentic messages about DMOZ and the efforts of our volunteer community.
• Highlight enhancements, both current and future.
• Allow editors to showcase their categories and describe, in their own words, why DMOZ is so important.
• Recruit new editors. If you have access to the Web and are passionate about a category, find out how to apply.

But most importantly, they want to use the blog to get feedback from the community.

The Managing Editor, Bob Keating, said:

What do you think about DMOZ? Why do you use the directory or data? Is there something you would like to see fixed? When you’ve been around as long as we have, some people are bound to have great things to say, while others might have a few choice words based on their personal experience. Either way, we want to hear it :)

Check out the new DMOZ blog at http://blog.dmoz.org.