Calacanis made his remarks during an “Ending The Content Arms Race” talk at FM’s Signal LA event. My semi-live blogging of his talk is below.

Jason recaps how he started Mahalo in 2007 reaction to what he considered bad search results on Google, results that were filled with spam. He felt human editors could do a better job selecting results for the most popular searches. Along the web, the company transformed into providing content. But last year, as he felt the Mahalo was heading “sideways,” he revisited its mission.

The Content Explosion, But Where’s The Quality?

Mahalo found that people really wanted articles, video and Q&A content, so it began to focus more strongly on that. But it also found it was playing catch-up against companies that were cranking out huge amounts of content.

Demand Media does 5,700 pieces per day; Aol 1,700 per day; Yahoo’s Associated Content at 1,500 pieces per day. All this versus Mahalo’s 1,100 pieces. The figures are all off a slide Calacanis flashed up — I don’t know the source materials for those figures, sorry. The slide above is from later in his presentation, when he shows the figures again but says less content might be better.

But in trying to fight in punching out more content, Calacanis also realized he was generating bad content. He flashed up a slide of Mahalo’s “How to Play The Xylophone” page which started out as step one, “Be Sure You Want To Play The Xylophone,” as you can see below:

Why was he putting bad content like this out, he asked himself? [Part of the reason for him asking was because last week, Business Insider did a send-up of that page -- the screenshot above comes from that post, which goes into more detail about the former page].

Why were his competitors doing the same? He flashes a pretty bad Yahoo Answers page that gets laughs; a page from eHow on how to bathe a small rodent:

Calacanis also pokes at the Huffington Post. After explaining he knows Arianna Huffington, has talked to her in the past, likes her — he brings up an example of the Huffington Post summarizing someone else’s news articles. He says 80% of the Huffington Post is simply rehashing of other people’s content like this.

“It’s mind blowing to me when I see the Huffington Post beating the people who are doing the original reporting,” Calacanis said.

The Google Giant Awakens

But the days of riding to the top of Google are over, he says, flashing up a blog post from the head of Google’s web spam team Matt Cutts, who said in late January that Google would looking at ways to to prevent “shallow or low-quality content” from doing well in its search results:

People are asking for even stronger action on content farms and sites that consist primarily of spammy or low-quality content. We take pride in Google search and strive to make each and every search perfect. The fact is that we’re not perfect, and combined with users’ skyrocketing expectations of Google, these imperfections get magnified in perception. However, we can and should do better.

Says Calacanis, this was a sign that Google was awakened to content factories, bad news for those businesses.

“The one rule of working with Google is don’t make them look stupid. If you make ‘The Google’ look stupid, they’ll f- you up,” he said — except the audience got the full F-bomb. “eHow, you’ve awoken a giant,” he went on — explaining how he’s been on the wrong end of the Google giant in the past.

Calacanis went on to say he likes Demand Media, knows executives there as well as at other so-called “content farms” and has lots of admiration for them. But they all need to change, and the solution to calming the giant is for the content to get better.

Stop The Internet Pollution

“We have to look in the mirror and ask, ‘Is this what we want create for our users?’ We are polluting the internet.”

Now Mahalo spends $100s to $1,000s on each page to ensure better content, Calacanis says. “It’s good for business if we get back to focusing on quality … we have to switch.”

Calacanis shows a “How to cook a turkey” page on Mahalo which he says has tons of good video content, the decries it has to compete against 17 different articles from eHow on every variation of how someone might want to cook a turkey. “Do you guys understand now why I’m going insane?”

Going on, he said: “We’re going to make high quality content, and I will ride Google every day until I’m number one for ‘How to cook a turkey’.”

Calacanis also asks the advertisers at the event not to support poor content. “You shouldn’t put your ads next to sub-par content. We will not make content unless we have an expert. Demand Media will make content if someone will take $10.”

He warns again about Google: “Google is figuring it out. eHow makes Google look stupid.”

Last week, Calacanis says the Mahalo Xylophone article got updated, at the cost hundreds of dollars, with lots of videos. “But I can sleep at night again.”

Looking at the page myself quickly, I see what appear to be two original videos. I don’t know how much more content has been added overall compared to the old article. I suspect no single page is really going to explain how to play any musical instrument in any depth.

Calacanis concludes with a question to his competitors: “Are you making content we can be proud of or pissing off Google?”

See also: Demand Media & Being “The Best Click” On Google for a related article from the event today.

Cuban’s speech at the OnMedia conference was reported by several publications. I’ll quote from AdWeek:

“Google is a vampire, and you run scared,” he said. “There is no reason to be indexed in Google.”

And:

He said readers who find headlines via Google rarely convert to traffic, and publishers have a hard time monetizing that traffic.

“You haven’t gotten anything back except that you’ve turned into zombies,” Cuban said.

I’ve bolded the key parts that I’ll contrast against Cuban’s backing of Mahalo, which is supposed to be a human-powered search engine. Both Mike Arrington at TechCrunch and Mathew Ingram at GigaOM have counterpoints to Cuban’s speech you may also wish to read.

Mahalo: A Mark Cuban Investment

From the Mahalo press kit, which you can find via the Mahalo advertising page, we learn that Cuban is an investor in Mahalo:

Mahalo: It Loves Google, Even If Cuban Doesn’t

The press kit also teaches us other interesting things. We discover that Mahalo provides “high SEO value,” as you see stated here:

For those new to the acronym, SEO stands for search engine optimization, the practice of generating traffic from the free listings that search engines provide. As Google is the biggest of the search engines, anyone doing SEO is largely tapping into Google.

This means that Cuban is an investor in a company that does NOT see Google as a vampire. Quite the contrary. Mahalo sees Google as a valuable traffic resource. Indeed, another slide from the press kit has Mahalo bragging about how it ranks at the top of an actual Google search results page (as it still does when I looked today). We’re told how Mahalo pages “rank highly in search queries” as part of the slide:

Rather than inform advertisers that Google is a vampire to be blocked — Cuban’s advice, remember? — Mahalo touts the opposite, how it can “help our partners increase their search engine rankings,” as you see in this slide:

Indeed, Mahalo bills itself as quite the SEO shop, able to do keyword research and organic linking for clients — I mean advertisers:

So let’s go back to those parts I bolded from Cuban’s speech. No reason to be indexed in Google? You don’t get anything back? Mahalo — a Cuban investment — clearly believes the opposite.

Yes, Mahalo Does News

Ah, but Cuban was talking about news content! Those headline readers, they don’t convert for news sites. Things might be different for a non-news site like Mahalo.

Actually, Mahalo does quite a bit of news targeting. Let’s look at the current Mahalo home page:

I see news about the Oscar nominations, news about Justin Mentell’s death, Super Bowl 2010 news, news about the Grammy Awards. Lots of news. Mahalo — it’s a news site, among other things.

Look here, at the home page of the Wall Street Journal today, unquestionably a news site. I’ve pointed at four different news topics on it: Obama’s 2010 budget, the Toyota recall, the Haiti earthquake and the Apple iPad:

Mahalo has pages on all these topics:

Maybe Mahalo’s run by a bunch of idiots who are wasting their time going after these news topics to get traffic from Google. If so, Cuban — as a wise investor — should advise that Mahalo focus on things more likely to convert. Alternatively, Mahalo’s making money off these efforts and potentially, so might news sites with original content.

If Cuban Can’t Police His Own Aggregators…

Did I say original content? See, that’s another irony. This is also from Cuban’s speech:

Cuban dared newspapers to stop linking their stories to Google and to police other aggregators….

“Show some balls,” he said. “If you turn your neck to a vampire, they are [going to] bite. But at some point the vampires run out of people’s blood to suck.”

If that word “aggregator” is leaping out at you but doesn’t really mean much, check out my How Search Engines, Aggregators & Blogs Use News Content article. It explains what they are and how everything works. In short, an aggregator is a site that summarize content that lives on other sites.

Now let’s take a closer look at the Mahalo page about Justin Mentell’s death:

There’s a whole lot of aggregation going on there. If Cuban’s urging that aggregators be policed, perhaps he could start with his own investment – Mahalo. Add Icerocket to the list, too. That’s another Cuban-backed service which offers news search like Google does. Here’s Icerocket aggregating news headlines about Cuban’s own speech against search engines aggregating news content.

To really visualize how much Mahalo is aggregating, look at this illustration:

See that arrow? It’s pointing at the original content. It’s hard to find it on the page versus all the aggregated/scraped content that comes from other sources. The illustration, used with permission, comes from Aaron Wall’s excellent piece published last week, Mahalo SEO Spam Case Study. Be sure to read that article for a deeper look at some issues with Mahalo.

My, Mahalo, How You’ve Changed

Of course, not all Mahalo pages will be like the one above. Some of the news pages I mentioned earlier have short summaries about news along with some aggregated — but at least hand-selected — links.

Still, Mahalo seems a far cry from the human-crafted results it launched with. Consider this page about fires in Malibu, as it looked in 2008:

Today, it looks like this:

Most of of the curated links are gone. You get some fast facts, then a ton of the automated aggregation.

Conspiracy Time!

Now let’s put on our tin foil hats. Mahalo makes money tapping into breaking news topics. Cuban makes money off of Mahalo. Cuban advises media owners with original news content that they should drop out of Google, especially since those Google visitors are worthless. If they do drop out, who benefits from the spots that open up in Google? Potentially, Mahalo. Potentially, Cuban.

Nice advice.

Similarly, who else do we see as a Mahalo investor? Let’s go back to the Mahalo slides:

News Corporation. And what’s News Corp been saying about Google?

News Corp chairman and CEO Rupert Murdoch has questioned Google’s use of news content many times, saying it, along with other search engines and aggregators, just steal his content and engage in theft.

News Corp owns the Wall Street Journal, whose publisher Les Hinton called Google a vampire seven months ago, before Cuban seemingly borrowed that metaphor.

News Corp chief digital officer Jonathan Miller said last November that visitors from Google are the “least valuable” to him — another theme added to Cuban’s speech.

How Might News Corporation Gain?

Who benefits if media owners take News Corp’s advice and pull out of Google? Mahalo, potentially — which News Corp has a stake in. Mahalo, by the way, which was founded by Jason Calacanis who used to work for Miller, counts him as a mentor and perhaps was used to float a trial balloon about the Wall Street Journal considering an exclusive deal with Bing.

Alternatively, as I covered in Garlic For The Google Vampire last year, News Corp benefits if media owners bail out of Google because it, unlike most other publications, probably has “must carry” status. As I wrote:

The Wall Street Journal and the Associated Press carry enough weight — are arguably “must carry” publications — that they’ll probably land some Google cash, in the end. And Google won’t call it pay off money, when it happens. We’ll get a euphemism about the deals being done for “new” or “different” uses rather than the “right” to list links.

Other papers and organizations, which struggle much more than the WSJ or the AP, won’t get anything. That’s why I don’t think Google should do such deals secretly. More on that in my Open Letter To Google & The AP: Reveal The Licensing Terms article.

Back To Reality

Now take off the tin foil hat. I’ve painted a possible conspiracy theory. I don’t believe it. I don’t think News Corp has taken the stance it has in hopes of driving out all other media outlets. I actually think News Corp, like many news organizations, has some serious concerns about how news can thrive in a time when fair use means “anything goes” to some. I also think the company, like many other media organizations, reacts badly to legitimate fair use out there because it has yet to learn how to thrive in the digital world.

As for Cuban, I don’t think he made his speech as part of an uber-plan to help his Mahalo investment. I think he probably has some heartfelt views that content owners are somehow being ripped off. From November, my If Newspapers Were Stores, Would Visitors Be “Worthless” Then? post covers more about that, including comments back and forth between Cuban and I on the topic.

Enough Hypocrisy & Rhetoric

What I don’t like is the hypocrisy both News Corp and Cuban have shown on this issue. A Look At All The Sites Owned By Rupert Murdoch That ‘Steal’ Content from last November at Techdirt covers some of the aggregators that News Corp runs, despite its anti-aggregator stance. My article today illustrates Cuban’s backing of aggregation at Mahalo plus touches it at Icerocket.

The lines people want to draw on both sides of the fair use and aggregation debate aren’t that black and white. Nor does the rhetoric bring anyone closer to solutions.

Postscript: Mark Cuban has posted a response to my article here that I’d encourage people to read. A few comments from me about it.

The other day in New York I gave a speech at the AlwaysOn Conference …. when I speak to a group like this, rather than just shilling a product, service or position as many …. I try to put myself in the business shoes of the audience. Then I discuss what I would do if I owned, ran or invested in their business, and the approach I would take to some of the strategic issues of the day.

The concept of directing comments to a vertical segment of a market is nothing new. I have been doing it for more than 20 years. Yet for some reason, based on comments from a few folks over the past couple days, there are some relatively high profile people in the internet business that have a tough time grasping that concept.

I have no problem grasping that concept. There were parts of the speech where Mark made suggestions about what he would do that fit right into this. But he also made declarations about the value of search traffic that just frankly seemed wrong. That’s why he’s getting the reaction.

Look, let’s say Mark were speaking in front of an audience that didn’t believe in vaccinations as a way to prevent illness. If he then started declaring that vaccinations were unsafe, just to put himself in the shoes of those in the audience, plenty would have spoken out against it.

It’s one thing to speak to an audience’s concerns. It’s another thing to position yourself in believing and backing some of those same concerns. To date, Mark’s given every indication that he believes Google is a vampire that sucks publishers dry. He’s done that in some of his writings where he’s not speaking in front of any particular audience but, I assume, just speaking his mind to voice what he believes in a topic generally.

Further on, Mark writes:

I think he Danny likes to banter to create traffic, smart on his part. But I also think he doesn’t fully understand all the business elements on some of the topics he has challenged me on.

Sure, I love traffic as much as the next person. But the most important thing to me is that important issues are covered, and that there is some balance where it needs to be.

I wrote my piece in response to Mark between midnight and 5am the day it was written. I simply couldn’t sleep after pondering his statements and having seen the Mahalo press kit recently. I wasn’t kept up because I thought “Oh, goody, I’ll get a lot of traffic out of this.” I was kept up because I love journalism, I know search, and I deeply hate when people spout off stuff that I find massively incorrect to media owners who desperately need better advice.

I don’t understand the business elements I’ve challenged Mark on? Well, for one thing, I came from a newspaper background. Five years working for daily newspapers. Not on the business side, nope. But that’s about five more years of newspaper experience than I believe Mark has.

I left papers because while I wasn’t on the business side, I could see clearly that they were going to fail when it came to the web. I watched my own paper at the time, the Orange County Register, struggle to decide if it should do AOL, Prodigy, CompuServe or something else. I jumped out in 1995 to do web development with a friend, instead, knowing it would be the web. Side note: the fact the OC Register has its domain name right now was due to our web development company. We registered it for them helped drag them a little bit online.

The web development company didn’t take off, so I struck out on my own as journalist. From literally nothing (and I do mean nothing, there was a time when I debated if I could by a London Underground transport card for an extra zone because of the extra $1 expense), I built a web site entirely around search. One that offered both free content and material behind a paywall, which helped it survive the dotcom downturn. A giant conference series along the way.

That all came from me seeing that search was a story, and that it was a story that deserved more than a once-a-year article about it that was commonplace for the magazines that I freelanced for in 1995. I thought we needed our own CNN of search news. So I started that businesss, a business that any media publisher could have easily have. And in 2006, the entire thing got sold for $43 million. From nothing. That’s not the $6 billion that Mark got from his Broadcast.com sale. But it was a fair chunk of change for editorial content that literally came out of nothing.

At the time of that sale, I no longer owned the original properties, so I personally didn’t come away with those millions. Darn! And I couldn’t work out a deal with the new owners, so I left and started over. Everything rebooted, from scratch. Enter new site here at Search Engine Land. Entire new conference series, Search Marketing Expo. Our Sphinn news sharing site, as well as our Search Marketing Now webinar series. All through my own Third Door Media company.

All this was done without needing VC money. No various rounds because we actually generate revenue to pay our own way. Last year, in the worst economic downturn in the United States since WW II, we generated a small profit. All from editorial content. News content and information. From visitors we gain through social media marketing, email marketing and search marketing. Google — you know, that vampire.

So yes, I think I understand some of the business elements of running a news operations in the days of the internet. I also know many news SEOs in the newspaper industry who understand the traffic they bring in. I understand how fundamentally screwed up the newspaper industry is when those same SEOs can’t get their technical departments to implement a friggin 301 redirect without debates and justifications. As I joked on Twitter last year:

for want of a 301, a newspaper was lost. or a website. tech folks, give your SEOs the 301 redirects they want.

I’ve also seen newspaper executives — the people who are supposed to save the industry — say things that make me desperately afraid they really don’t understand how search engines work. And if they don’t understand these things, it’s hard for them to make the right business decisions. That’s one reason I invested a huge amount of time on a three part series with Google last year about how it deals with news content. It was important information to get out. It was good information to get out. And I can only hope some of the actual business people paid attention to it.

As for Mark’s hypocrisy telling media owners that Google’s a vampire when he has investments in similar vampires, he wrote:

Danny Sullivan thought he had caught in some hypocritical act because I am an investor in Mahalo, a human powered search engine that leverages SEO techniques to increase traffic and revenue. First of all, I invested in Mahalo in 2006 . Not yesterday as Danny would seem to imply

I didn’t imply that it was yesterday. And I don’t care when he invested in it. He’s still an investor. He seems to disagree with how news aggregators and search engines list media content. Rather than speaking generally to what he thinks media owners should do about the vampire problem, he could use his influence to prevent it within his own investments. And if he’s not listened to, get out of them or speak publicly against them.

Further, he writes:

In the case of Mahalo, unlike newspapers, they are making good money from Google traffic. No reason to stop doing that. On the flipside however, its fair to point out that Mahalo does use some newspapers content to support their content. If a newspaper would ask me if they should block Mahalo, the fair answer would be that there is no reason not to.

Unfortunately, those newspapers can’t block Mahalo if they wanted to. Some of those links are added by human editors. Is Mark suggesting that newspapers should constantly be monitoring Mahalo to see when new links are added, then asking to have them removed? Or has Mahalo gained some type of universal “never link to this site” to advise editors, that I’m not aware of.

As for the automatic links, they come from tapping into other search engines such as Google, YouTube and likely some other places that I can’t quite figure out yet. To block Mahalo, you have to block those places. Mahalo, unlike a real search engine, has no spider that can be blocked automatically using a robots.txt file.

Mahalo Answers is the newest entry into the crowded Q&A reference site space, but it offers a twist that its biggest competitors don’t: the chance to earn money by contributing to the service. Mahalo founder Jason Calacanis says the new service is the third and final piece of his original vision for Mahalo, making it a site that combines search, content, and knowledge exchanges.

Yahoo Answers is the 800-lb. gorilla in this field, with some astonishing numbers reported earlier this year: 135 million users and 500 million answers worldwide, and growing at a rate of 11 million new answers per month just in the U.S.

What Calacanis hopes will set Mahalo apart is cash. Users asking a question can offer money as an incentive to attract good answers, and after the fact, other readers can also tip the best answer(er).

Mahalo Answers also brings money into the Q&A equation by giving experts the opportunity to charge money for direct questions. An SEO expert, for example, could set up shop in Mahalo Answers and charge a small sum to anyone who wants to ask a direct question.

Money was part of the now-defunct Google Answers ecosystem, which functioned very similarly to what Mahalo is doing now. The primary differences are that Google Answers required a cash payment, while it’s optional on Mahalo, and questions on Google Answers were asked to pre-screened experts, while anyone on Mahalo can answer a question.

Yahoo Answers also has an open approach with any community member being able to answer questions. But in his blog post about today’s launch, Calacanis takes a swipe at Yahoo Answers’ free-for-all approach:

“…the chance of making some money in Mahalo Answers is obviously a lot better than the guarantee of making none in Yahoo! Answers–not that I’m comparing the two products.”

But everyone else is, as evidenced by the discussion on Techmeme.

The Search Evolution So Far

Before going ahead, let me summarize what I covered in my past article, in terms of how search engines have changed over time to create and rank the results you get when doing a search:

• Search 1.0 (1996): Pages ranked using “on-the-page” criteria
• Search 2.0 (1998): Pages ranked using “off-the-page” criteria
• Search 3.0 (2007): Vertical search results blended into regular search results

The evolution above is not perfect. For one thing, some “Search 3.0″ blending started to happen years before 2007. It’s just that in 2007, I felt all the major search engines made the leap into Search 3.0 in a significant way.

As for Search 2.0, looking at off-the-page criteria such as links, Google kickstarted that heavily in 1998. However, some link analysis happened before then, and all the major search engines probably didn’t get on board to using it more fully until 1999-2001. But the launch of Google in 1998 remains the benchmark year in my mind, for that particular change.

The evolution is also only applicable to crawler-based search engines, those that use automation to gather web pages, store copies of them and search through the compiled index to create listings for searches. Yahoo was a major player using human power before 1996 and continued this way for years. Indeed in 1999, a majority of major search engines were presenting human-powered results. This quickly changed as Google grew. Yahoo made its human results “secondary” to crawler-based ones (then provided by Google) in October 2002. Today, all the major US-based search engines depend on crawler-based results.

To cap off the caveats, the evolution above is not the only way search engines can evolve. That’s just how things have largely gone with US-based search engines, which in turn tend to also be the major search engines for most countries around the world. There are exceptions. For example, Naver is the dominant search engine in Korea — and there, listings are largely human generated.

Search 4.0: The Human Factor

Onward to Search 4.0! As I said in my opening, to me this is the move for search engines to make use of human data as part of their ranking systems. In particular, it means human data generated by you, by those you know or by human editors.

Search engines already make use of some human data. All the major search engines, for example, monitor what we click on within the search results. This helps them determine if a particular listing is drawing more or less clicks than would be expected for the position it holds. For example, if the number two listing for a particular query is getting less clicks than “normal” for a listing in that spot, perhaps it’s a bad quality listing that should be replaced with another.

Another example: all the major search engines make heavy use of link data — and that link data is largely human data, humans both “voting” with their links and “tagging” pages by the words they use in the links. Google Now Reporting Anchor Text Phrases and Google Kills Bush’s Miserable Failure Search & Other Google Bombs provide more about how links are used in this fashion.

When I talk about putting human data into search results as part of Search 4.0, I mean things that are more aggressive or active than what I’ve covered above. I’ll start off with the most refined Search 4.0 implementation out there, Google’s personalized results.

Google: Search 4.0 Gets Personal

With Google Personalized Search, the web pages you visit, bookmark and things you click on within search results at Google are used to custom-tailor search results for you. The personalization is not as dramatic as with a place like Amazon, where if you purchase a book once, Amazon seems to continually push similar books like that at you forever. Shifts are far more subtle, mainly to help elevate results from sites you frequently visit.

To understand more, these articles go into depth about the process:

• Google Ramps Up Personalized Search
• Google Search History Expands, Becomes Web History

I’m fairly bullish on personalized search as an important addition to other factors (Search 1.0-3.0) in improving results. For one thing — better or worse — people often judge the relevancy of search results based on ego searches. Does a search engine find your home page, blog and related material when you search for yourself? Does it find your company? Personalized search is an ego search reinforcer. Because you go to your own places on the web often, Google senses that you want them to show up higher in search results, and they do. It’s a genius way to ensure anyone reviewing the service comes away pleased!

Of course, fulfilling ego searches can also be an relevancy advancement, not just a marketing ploy. There’s an excellent chance you’d have better searches if sites you visit more often get a bump in the search results. Personalized search can do this. In addition, over time, personalized search can potentially figure out other sites that are similar to those you visit and give them a relevancy boost.

Since Google expanded personalized search last year, there’s been one further major development. Personalized search uses searches over time to refine results. However, Google also has a system it is testing to refine results based on the last query you did, even if you aren’t taking part in the personalized search program.

“Previous Query” Refinement Coming To Hit Google Results explains more about how this works. It’s been used to improve the ads shown on Google for almost a year now, and it’s currently being tested to refine regular results. Google said that previous query refinement has been one of the strongest signals on how to personalized results so far.

Social Search: Promise Or Hype?

Last year, blogger Robert Scoble kicked off a round of “Facebook’s gonna kill Google” with a series of videos suggesting that because Facebook knows who your friends are, they’ll be able to apply that “social graph” data to improving search results.

The Promise & Reality Of Mixing The Social Graph With Search Engines was my response, a bucket of cold water explaining that using social data wasn’t some new idea that had never been tried before. The article went into depth explaining how Eurekster and Yahoo both assumed search could be “socialized” similar to photo sharing or bookmarking, only to find that wasn’t the case.

Yahoo had little take-up of its social search product. I’ve never seen the company explain why. My own suspicion is that take-up was low because search is NOT a social activity. I believe people tend to search when they have an immediate desire that needs fulfilling, and taking time away from the search activity to “share” with others is a distraction. Consider the person who has a broken water pipe. They might search quickly to find a plumber. They aren’t likely thinking at that moment that they want to tag and classify the search they conducted, much less the plumber they called. They just want the pipe fixed!

Eurekster has said that it found social activity worked better when people organized to build what it calls “Swickis,” search engines that hit only a custom collection of web sites related to a particular topic. Earlier this year, Eurekster formally came out of beta. However, the service has been entirely off-line for almost a week now. Practically no one has noticed, which speaks volumes to its usage and that aspect of the social search potential. Twitter, which some still view as a niche service, can hiccup for an hour and produce reams of blog attention. Eurekster goes silent, and the web stays silent about it.

I’m A Facebookholic & I Have 5,000 Friends

Still, couldn’t Facebook have more luck? For the record, when I spoke with Facebook director of engineering Aditya Agarwal about social search ideas last December, he was far more realistic than outsiders who hype what Facebook could do. In particular, he wasn’t certain how useful the social data actually would be for refining web search.

I plan to do a future article with Agarwal to explore this more. As a reminder, Facebook right now has no web search feature at all. And while it does have an ad deal with Microsoft, our previous Microsoft’s Facebook Ad Deal Doesn’t Include Search article covers how a search partner hasn’t been selected.

Let’s assume that Facebook does select a search partner, which it will need, since trying to index billions of pages and serve millions of queries each day is not an easy task (just ask Microsoft what it’s like to build that from scratch). What could it do with social data?

For one thing, it could monitor what people are clicking on in a potentially more “trusted” environment. Anyone can use web search anonymously, even sending in clickbots to make it seem like some particular listing is super hot. Having to register to be in Facebook and search from within there might make the clickstream data less noisy. But then again, it’s still a fairly open door that someone can walk through, if they want.

Facebook could tailor results based on what friends are searching on. If it knows what you and your 25 friends all seem to select from results, it could ensure those sites get ranking boosts for future searches. That’s very similar to personalized search, except it sounds full of extra friend-goodness, right?

The flaw here is plenty of people have friends on Facebook they don’t know. Some people collect friends for fun (and profit). Some people get friended by others just looking to build up their profiles. Some people you might friend not because you like them but because it’s easier to friend them than say no. Any of these instances can cause “pollution” of the social data that supposedly was going to improve your search results.

Consider also the case of someone who might work at some very conservative company but outside of work is a freeliving, devil-take-all person. Do they want coworkers who are friends to flavor their search results or those friends they hang out with when work is over?

Finally, privacy is an overlooked issue when it comes to social search. People often search for intensely private, personal things using search engines. Search engines are almost like confessionals, where people seek solutions to problems they might not tell real people that are close to them. With social search, do they have to remember to turn off a sharing feature that might be activated by default? And if it’s not on by default, will it get any take-up at all?

In the end, I think there is some potential to tapping into a social network and applying it to search. However, I still remains uncertain how that will unfold. It especially remains uncertain that this is somehow the secret sauce for anyone to jump past the current state of search.

Return To Humans: Hello Mahalo!

Earlier, I’d mentioned how Yahoo started off using human beings to create its search listings in the days before Google existed. Over time, the human soul in search was lost to reliance on the supposed scalability of machines. Anyone who wants to see how much we’ve handed over to machines need only search for buy cialis online on Google. At the moment, the results are littered with online discussion forums that have been “borrowed” by affiliates and others hawking deals.

Those pages will sit there for a day or two or three or potentially weeks, as Google usually tries to find an algorithmic solution to getting rid of them. The idea is you might have to suffer a bit in the short term until a long-term cure is found. But then like a virus that mutates, something else gets through, requiring a new long-term cure.

Enter humans. A human editor, reviewing results like that, can immediately spot junk that should get yanked. Even better, a human editor could act as a curator. How hard can it be to find 10 quality sites that should come up for that or other terms?

That exact human solution, of course, is what Mahalo has been banking on. Mahalo, launched last year, uses human editors to hand-pick top results. For background on the service, check out these past articles:

• Mahalo Launches With Human-Crafted Search Results
• Mahalo Greenhouse: Get Paid For Writing Search Results
• Mahalo Follow: Toolbar Gives You Human-Powered Alternatives To Searching, Surfing
• Mahalo Adds The Social Graph To Search
• Mahalo Adds More Social Features

As part of a talk I do on Search 3.0 and Search 4.0, I have some screenshots from last year that illustrate well how a human can indeed do better than the machines, for some queries. Remember the fires in Southern California at the end of last year. After a series of wide ranging ones, Malibu was hit with a second one a month later. Here’s what those searching on Google got in response:

The news box at the top is great, but sometimes searchers skip past things like this and go to the first “real” result. That’s a story about the Malibu fire early in 2007, not at the end of the year. Other results were largely about the fire of October 2007, rather than November 2007 (which is what many searchers at the time I snapped this would have been interested in).

Here’s Yahoo:

Again, news results at the top, then unlike Google, places you’d expect to find news about the fire — the local paper; ironically a map of the fires on Google Maps that Google itself didn’t return; the Malibu city web site, as well as the fire department.

Here’s Microsoft Live Search:

Very similar to Yahoo — a news box, the fire department, the Red Cross. What’s not to like? Well, let’s look at Mahalo:

Note at the top that Mahalo’s human editors understand there’s a different fire that happened in the past, in October 2007, and offer a link to a page about that. Then there’s a nice list of news sources, followed by coverage by date. Over to the side, a synopsis of the current situation. If you could see more of the page, there was lots of other categorized information.

It’s nicely done. It’s very helpful. And it was created with a human thinking about what other humans might want to see, rather than machines guessing.

Scaling Humans

So is Mahalo founder Jason Calacanis onto the Google-killer, human crafted results? No. I think human review can be part of the solution, part of the Search 4.0 addition to what we have out there already — but humans can’t craft pages for every possible search. In addition, it’s hard to keep those pages maintained once they’ve been made. It’s also easy to cross over from being a search resource that points to other resources to becoming instead a destination site. I think a good search engine avoids that (and Who’s Ranking For Knol? Hello, Wikipedia! has more on this topic).

Mahalo can also be overwhelming. Try a search for hillary clinton and there’s category after category. Background links. News links. Photos. Videos. Bio links. Blogs and message boards. Plus, there’s even more. I think at some point, you want your search engine to make some key choices for you, not flood you with so many that you don’t know where to begin.

Another issue is that what Mahalo’s human editors do, machines can get close to. Hakia especially stands out here. Search for hillary clinton there, and you’ll see how listings are grouped into categories like Awards and Biography without humans being involved (and see Social Networking Through Search: Hakia Helps You Meet Others for background on how Hakia works).

More Humans

There is another major search project involving humans: Search Wikia. Backed by Wikipedia cofounder Jimmy Wales, the service aims to involve humans in rating pages, annotating them and helping determine the ranking algorithm for choices the machine side of the project makes.

The articles below have more background on the service:

• Search Wikia: Not Even A Remote Threat To Google
• Search Wikia Adds Alpha 0.2 Features & More

Right now, the quality of the service is poor, as Search Wikia itself readily admits. There’s still lots of work to be done — and even with it, it might never succeed. But allowing humans into the process is, in my view, a good thing.

Indeed, even Google understands this. Last year, Google started doing some education about how human “signals” are already incorporated into its algorithm (see Google’s Human Touch and Google & Human Quality Reviews: Old News Returns). Aside from this, last year it also started testing a way for people to annotate search results — add those they like, remove some, suggest other ones.

Watch Personalized Search

Overall, there’s a role for humans, a way for them to be in the search process to enhance results. Actually, there will be several ways for them to be involved. Exactly how remains to be seen, of course.

Of the things I’ve outlined — personalized search, social search, human editors — I think personalized search is the one that will emerge as the major part of Search 4.0. That’s not to discount other things being tried, and they’ll contribute in some ways. But to me, personalized search has the most potential for another big relevancy leap. We’ll see!

Below you can hear Jason’s presentation yourself, then you can hear the Q&A portion that covered search marketing and human powered search. Note that the video production could be better. Hey, I just got a Macbook — it’s my first time playing with what it can supposedly do. Do don’t hassle me over the titles that could be better. Also, I will get the entire session up with the presentations from Jimmy and Steven, along with the further Q&A. But first, Jason:

The Q&A:

And here’s some of the reaction to initial reports of his comments, before anyone not at the show could hear what he said in full:

• My Plea to SE Conferences: Turn Calacanis Away, Eric Lander
• My Plea to SE Conferences: Turn Calacanis Away, Sphinn discussion on post above
• Why Calacanis Should Not Be Allowed to Speak at Conferences, Slightly Shady SEO
• Why Calacanis Should Not Be Allowed to Speak at Conferences, Sphinn discussion on post above
• Jason Calacanis has pissed off SEO’s… again. Seriously., Sphinn
• Srsly are SEOs Really That Easy To Manipulate?, ShoeMoney

And here’s further coverage of the show (for initial coverage from the first day, see SMX Social: Mahalo To Do Microformats, Search Wikia Adds Alpha 0.2 Features & More):

• 20 Take-Aways from SMX Social Media – Scott Clark, Scott Clark
• Evangelist – The Marketer’s Role in SMM, BruceClay.com
• Micro Communities, BruceClay.com
• Wikipedia, Yahoo Answers & Answer Sharing, BruceClay.com
• Social Media Expo (SMX) Report from Long Beach, California, Wpromoter
• SMX Social Media Conference in Long Beach, MindCradle
• Rand Fishkin | Jason Calacanis | SMX Social, Gary Pool
• SMX Social Media Coverage Round Up, BruceClay.com
• SMX Social Media Pics, Scott Clark @ Flickr

Jason Calacanis of Mahalo, Jimmy Wales of Wikia Search, and Steven Marder of Eurekster all spoke at the end of the day on our Social Search: The Human Challengers panel. Lisa Barone provides live blogging coverage in her Keynote – Social Search: The Human Challengers post at the Bruce Clay blog.

During that session, Jason shared that Mahalo is providing microformat support. Mahalo Adds Microformats from Mahalo’s Sean Percival covers how this works, complete with screenshots. Check it out! See also further discussion on Techmeme.

Also during the session, Jimmy Wales shared that Wikia Search has added several new features as part of a new alpha release, one he dubbed "Alpha 0.2." Wikia Search doesn’t seem to have news up yet on its own site, but Wikia Search Launches Major Enhancements to Search Alpha from CenterNetworks covers how you can do things like preview pages and copy sections of them into search results. You can also add related search terms to queries, add pages you think should be added to a search result, and delete material you think doesn’t belong.

Scary, freaky? You bet — but Jimmy also covered how, like Wikipedia, the history of all this can be seen, which potentially provides some self-policing. Try the new features on the test site here. And further discussion can be found on Techmeme.

For other session coverage from the conference, see:

• Social Media Marketing Essentials, BruceClay.com
• Linkbait – Chumming for Traffic on Social Media Sites, BruceClay.com
• Extra! Extra! The Social News Sites, BruceClay.com
• A Marketer’s Guide to Social Bookmarking, BruceClay.com

We’ll also do a Day 2 post tomorrow with further coverage and add to it anything we’ve missed. Also, you can keep up with live blogging both at the Bruce Clay blog and via Twitter. Yep, Twitter. Michael Gray is diligently microblogging on our SMX twitter account.

In addition, Mahalo now lets you create “stub” pages (much like you can on Wikipedia). Jason Calacanis assures us on his blog that he has 50 full time people watching user-contributed content for spam and will not only remove spam, but might ban the domain and user as well.

For the purposes of this article, I decided to try the Mahalo Follow toolbar. I don’t generally use Delicious, and I’ve never used Ma.gnolia, so the cross-posting features aren’t as useful to me as they may be to others. If I could add the services I do use, it might be more interesting. I also didn’t love the fact that I had to give Mahalo my Delicious and Twitter login information.

The sidebar tips are potentially useful (at least until you have the ones you need memorized), but the sidebar itself takes up a lot of space. I also recently installed the Remember the Milk add on for Gmail, which shows up as a right sidebar, and I think this may be causing the Mahalo sidebar to have a funky layout in Gmail.

I also admit that I don’t get the sidebar in general. The help page says that it shows Mahalo content that’s related to the page you’re viewing. However, when I was viewing the Ma.gnolia page, the sidebar showed me how to make a dirty martini (perhaps because it picked up the word “mixed” in the “mixed martial arts” heading on the Ma.gnolia page. When I view my blog, it shows me content about Twitter, Biz Stone, and batteries, and I suppose that looking at my recent blog posts, that’s more relevant than anything else, but still not necessarily super useful.

As with much of Mahalo content, the best results seem to be that which are hand-picked by human editors, such as the new sidebar tips.

It remains to be seen how popular the new stub pages will be. Wikipedia has proven that users can get passionately involved in creating this type of content, and in the case of Mahalo, the pages will be overseen by Mahalo-employed editors, rather than volunteers, which may increase accuracy and objectivity. It’s not clear what advantage these pages have over existing sites such as Wikipedia, though (both in terms of what would motivate people to create them and to use them).

Blekko

What we know so far about Blekko isn’t much, and TechCrunch has the most details in its The Next Google Search Challenger: Blekko post from yesterday. Apparently Rich founded the company in September 2006, along with five other former Topix employees, after he left Topix in June.

Rich told TechCrunch not to likely expect anything public until 2009. I agree with Michael Arrington at TechCrunch that Rich has a track record that makes him well worth watching. The Open Directory was an initial success, though the model didn’t scale well. Some of that was within the founders’ control but had more to do with AOL’s lack of backing. The company should be dragged into the International Court Of Search Crimes and be forced to sell the ODP to someone who will support it properly. Topix has built a reputation and is still standing and succeeding — though I’d say it still has far to go to seriously threaten Google or Yahoo.

Rich adds a bit more in his Why Search? post today:

Having just spent 5 years in the media space, I’ve come away with the idea that editorial differentiation is possible. But the editorial voice of a search engine is in the index…so it has to be algorithmic editorial differentiation.

So far, it doesn’t sound like a social networking play like some of the others. We’ll be watching, Rich. Also see discussion today on Techmeme.

Powerset

Powerset is now a classic example of why you WANT to be a stealth start-up and say little. That’s because when you get too much early press — in part through your own doing — then fail to deliver anything, the hype can swing back at you hard.

The company came to light back in October 2006 via VentureBeat, with the twist being that natural language search would be the way forward. That caused me to write a long rant about the hype of natural language search in reaction. From the top of that:

This is a rant. It’s a rant from over 10 years of watching people trot out natural language search as the “killer” solution to the current state of search, something that’s happening once again with Powerset. That’s a search engine you can’t even use at the moment, but the hype will no doubt continue. To counteract that, my thoughts on and some history about natural language search.

Natural language search makes a compelling pitch for those who really don’t know search or haven’t heard the natural language mantra before. I’ve seen the pitch time and time again. You:

• Pick out an example that shows how “bad” search is on an existing search engine
• Demonstrate how natural language search would work better on your service
• Sit back and collect the press attention

I then went on to detail how natural language search had been hyped and tried over the years. The short story is this: It doesn’t take much natural language analysis to figure out what someone wants when they type in “britney spears nude” or “hotmail.” In addition, by and large I don’t believe enough people will change their basic search habits to enter long sentences when searching any time soon.

Since that time, we’ve pretty much had nothing out of Powerset other than the launch of Powerset Labs in September 2007. That launch hasn’t produced any cool applications that I’ve seen or heard about, nor much buzz. Instead, in November, we got a management shake-up.

For a more formal chronicle of the company’s developments, check out this area at VentureBeat and these search results at TechCrunch.

Finally, while I’m harsh above on Powerset, I actually had a long visit with the company in the middle of last year and was deeply impressed with the effort going on there. I’m still working on a long write-up to explain what’s happening. But in a nutshell, Powerset is trying to literally comprehend or understand each page on the web.

Today’s search engines don’t know what a page is about by reading words. They’re more or less doing pattern matching — finding pages that contain words similar to what you search for (or pages relevant to those words based on linkage). Powerset literally is trying to read and understand what a page is about the way a human reads a page and knows it is on various subjects.

I don’t see that as making it a better search engine that Google. Instead, I think it may eventually give it the ability to create a unique “auto-Wikipedia” style site, assembling knowledge pages on any subject automatically. I also think that there will eventually be some search benefit in comprehension of pages, but exactly how that will play out I suspect is part of being with an existing search engine and a more traditional model. With the array of patents Powerset has lined up, I suspect it will eventually get acquired by Google, Yahoo, or Microsoft rather than rollout its own product. But we’ll see.

Hakia

Like Powerset, Hakia has played the natural language search game. Unlike Powerset, it has a product anyone can use — live since at least the middle of 2006.

Again, I’ve been working on a long write-up on the inner workings of Hakia and have yet to finish it. It’s complicated, and I mainly want to cover what I find to be the real use of their technology — the ability to create custom “gallery” pages and understand those are related to particular searches.

It’s easier to show you what’s impressive. Search for hillary clinton, and you get a nice page showing news, her official site, biography pages, blogs & fan sites, news & interviews, and more. It’s very Mahalo-like, except it doesn’t require human editors like Mahalo and predates Mahalo by a year.

That categorization is something I know the major search engines could do, if they wanted. So far, they don’t. And so far, despite Hakia talking about its rising traffic, it has yet to make a serious mark. Moreover, in October, it made a serious shift to allow social interaction with its results. That’s a sign that the original plan that “natural language will win all” has failed to do so; therefore, another twist is needed.

Social Networking Through Search: Hakia Helps You Meet Others from Vanessa Fox here at Search Engine Land covers the change, plus it gets into the natural language indexing stuff I mentioned earlier that makes Hakia unique, plus has examples of gallery pages.

Mahalo

Credit to Jason Calacanis. He said he wanted to take on Google, then wasted no time getting Mahalo rolled out. OK, he also says he’s not taking on Google — just focusing on the top searches that he thinks would be better with human review. Sure, you aren’t taking on Google, Jason.

To date, Jason reports that Mahalo’s traffic is growing and strong. But to date, I’ve certainly see no webmasters taking about what a traffic driver Mahalo is. It would be early to call it a raging success, but it’s a nice alternative to have. Indeed, later this month I’ll finally finish my Search 4.0 piece that picks up from the conclusion of my Search 3.0: The Blended & Vertical Search Revolution article last November. I’ll show some examples of how the human element at Mahalo can and has kicked some Google and traditional search engine butt — though also how it isn’t the panacea some expect.

Some of our past coverage of Mahalo:

• Mahalo Launches With Human-Crafted Search Results
• Mahalo Greenhouse: Get Paid For Writing Search Results
• Search Spam Fight - Mahalo: 1; Squidoo: 0
• Mahalo Follow: Toolbar Gives You Human-Powered Alternatives To Searching, Surfing
• The Promise & Reality Of Mixing The Social Graph With Search Engines
• Mahalo Adds The Social Graph To Search

Search Wikia / Wikia Search

Wikipedia founder (as he prefers to be called; Wikipedia itself calls him cofounder) Jimmy Wales made waves a year ago when he said he’d take on “closed” Google with humans and a transparent search engine. Called Search Wikia (but, confusingly, it’s also called Wikia Search), Wales has grabbed attention from the press over the past year. Slamming at Google as a scary closed thing gets you good mileage, especially when you helped establish Wikipedia, a threat Google takes so seriously that it may launch its own Wikipedia-style site, Google Knol.

Now Wikia Search is at hand. A private “pre-alpha” test started in late December, an invite-only thing I still find odd for a service that’s supposedly all about the “transparency.” But on Monday, the general public will finally get a look at whatever Wales and his team have concocted. In the meantime, while Wales still hasn’t posted any news since July 27 to the “news” section of Search Wikia, press reports tell us so far:

• Only a tiny 50 to 100 million pages will be indexed at launch. The major search engines today have tens of billions of pages indexed. (AP)
• There will be a high degree of human editorial influence, though whether that’s over the algorithm or the search results on a per-query basis remains to be seen (CMP)
• An early screenshot suggested that Search Wikia might be evolving more into a Facebook-style service, perhaps with some ways for users to share results (Matthew Buckland & Wired)

Some of our past coverage of Search Wikia:

• Q&A With Jimmy Wales On Search Wikia
• Search Wikia Takes Steps To Crawl; Acquires Grub
• Search Wikia Gets Open Source Categorization Software
• Search Wikia Launches In 2007 With Private Beta

Cuill

Arguably the stealthiest of the stealth start-ups, Cuill (pronounced “cool”) has an impressive pedigree with its three founders: Tom Costello of IBM’s WebFountain project and Anna Patterson and Russell Power of Google’s TeraGoogle project, its massive search index. And last year, former AltaVista founder Louis Monier — who later went to eBay as its first eBay Fellow, then to Google — jumped ship from Google to join Cuill.

I talked with Cuill earlier this year to understand a bit more about what they are doing, but the details are still being held very closely. The main difference between Cuill and everyone else I’ve named above is that Cuill is founded by people who understand and have dealt with firsthand the challenge of indexing billions of documents.

Cuill recently took on more funding. Louis is also going to be doing a keynote at our SMX West search marketing conference, held in Santa Clara, California from Feb. 26-28. I’m thrilled to be having him since there are only a handful of people who have worked for the “old” Google (AltaVista), the current Google (when he was at the Big G), and a potential future Google (Cuill).

And The Winner Is…

If you think the future of search is on smart automation, Cuill’s definitely one to watch, and perhaps Blekko as well. If you think it’s the growth of humans, Mahalo and Search Wikia are your better candidates. The reality is that success will likely be a blend of the two. For the human services, a real open source index would be a big help — see Google: As Open As It Wants To Be (i.e., When It’s Convenient) for more about this.

But the reality is that all of these services will have an incredibly tough time to beat Google.

Google came along at a very special time, as I’ve long written. It had better technology at a time when all the search engines had abandoned improving search, since that was seen as a loss leader. The money was in portal features.

Today, search is a multi-billion dollar industry. If someone with a serious search threat comes along, you buy them (such as with YouTube), or you start to develop your own rival if it seems a real threat. Google’s not omnipotent — but you’ve already got a space where it’s Google, Yahoo, Microsoft, and Ask all seriously fighting it out (and the latter three, despite their funding and experience, still struggle against Google as being synonymous as a trusted search brand for most users).

To date, Google is the real exception of “a better mousetrap wins.” It’s far more likely the companies above, if they do gain traction, will end up being purchased for a large amount by one of the existing “search utility companies.”

Ah, sweet irony. Yesterday I spent some time raising concerns about knowledge aggregation sites like Wikipedia and the forthcoming Google Knol potentially ranking tops for every search conducted. Today, what’s in the top results for Knol? Yep — a new Wikipedia page on the topic!

The page was created yesterday and took less than 24 hours to show up. Looking at the top results for Knol is also fascinating in how until last week, the Google project wasn’t announced, so the results had no reflection of it. Today, they dominate the page:

Let’s run down the list.

1. KNOL is also the ticker symbol for Knology, and the Yahoo Finance page about that company has managed to hang on to the top spot.
    
2. The official Google Blog post on Knol comes next.
    
3. The official Google screenshot of an example Knol page shows up third. Some SEO advice to Google: Get a title tag on that page so it doesn’t look all weird when listed. You might also want to put a link at the top of the page over to your blog post so people hitting the screenshot have somewhere to go for more information.
    
4. Search authority Tim Bray warms my heart by covering how "transparent" Wikipedia actually is pretty closed given "a forest of acronym" and other issues that make me nod my head in violent agreement. But he doesn’t see Knol as a solution.
    
5. MarketWatch’s page about Knology comes next.
    
6. Mashable’s write-up on Knol is fifth. Sniff. We were one of the few places pre-briefed by Google on Knol and had an article with details not in the official blog post, which is what the Mashable article and virtually all other news stories were based on. But we get relegated to position 20 in the search results. Sniff. But congrats to Mashable, and we’ll look forward to when Search Engine Land is a bit older. With age comes authority and an easier way to make it to the top. We only just turned one!
    
7. Wikipedia makes it at sixth. It’ll be interesting to see if the page rises over time. FYI, Squidoo has three pages about Knol now. I like this one that’s just a copy of the official Google Blog post. I guess the author missed the Google copyright statement at the bottom of the post. The other two (here and here) are pretty basic. Mahalo’s got a nice page of mainly news commentary (though our write-up, sniff, isn’t listed). As for Yahoo Answers, three questions: how can someone get a Knol invite (here, and you can’t); are there reasons for Yahoo Answers folks to be afraid of Knol (here), and how does Knol compare to Wikipedia (here).
    
8. Noah Brier’s two paragraph summary of Knol pulls off a nice coup by getting into the top ten.
    
9. News.com’s write-up on Knol.
    
10. Wired’s write-up on Knol.

Also, I took a quick spin at Yahoo, Microsoft, and Ask. Main differences?

• Yahoo results are pretty similar to Google, though the Dutch Knol-Online makes it in the top results.
   
• Microsoft gets Knol Computers, Knol Farms, and Wikipedia’s Knol Tate page into the top results. Let’s hear it for diversity in search listings! You also get finance pages about Knology. As for Google Knol, you get one single page of ZDNet coverage — not even the official Google Blog post. Come on, Microsoft – that post should be there.
   
• Ask has even more diversity, from Ryan Knol Designs to this page that mentions someone named Knol, among many other people. Google Knol is covered by only a single News.com article about the service. The official blog post doesn’t show, and that’s just as disappointing as with Microsoft.

Jason acknowledges that Mahalo is a content site rather than a search engine. 30% of the site is editorially generated by their staff, but it’s presented in a search context. Mahalo contains about 26,000 pages so far and Jason says they’ll never cover the long tail of search. But he does think they’ll catch up with the amount of content on Wikipedia, and within five years should be covering up to half of what people are searching for. (Don’t worry Jason, it’s mostly Britney Spears and sex. And I see you’ve got those two subjects covered.)

Mahalo employs 60 full-time editors, 400 paid contributors, and 3,000 “Greenhouse” volunteers. This enables them to add around 1,000 new pages a week, but they need a way to find more quality links and gain more insight into what people want pages about. So, they’re now handing Tom Sawyer-like paint brushes out to the rest of us and looking to crowdsourcing to help paint the search results fence. Harnessing the wisdom (or lack thereof, as the case may be) of crowds helps them overcome what they feel is currently their greatest hurdle: adding and updating pages.

Anyone can now create a profile on Mahalo, then start recommending links for search terms (regardless of whether those terms have existing Mahalo pages). You can submit using either the Mahalo toolbar or directly from the site.

How will Mahalo combat spam? Each link will be reviewed before it’s accepted. You can see your accepted/rejected stats on your profile page. Accepted links appear in the “user recommended links” section of the search page, and anyone who contributes links will be listed in the “contributors to this page” section.

Mahalo won’t use a Digg-like system to enable people to vote on links. Jason says he’s looking for quality over quantity and doesn’t want users to simply skim a list of headlines and vote for what catches their attention. Instead, votes accumulate based on multiple recommendations.

All recommendations go to human editors, who either approve them, wait for further recommendations, or ban them. Is this scalable? Jason says he doesn’t expect a lot of participation (he cited the fact that most user-generated content tends to come from a small fraction of a community, and he expects Mahalo to be no different) and claims he’ll be happy with 50 people participating in the first month, with a few thousand by the first year.

However, he is looking at ways to scale this a bit, just in case he gets a few more than 50 signing up. For instance, they may implement a kind of “PeopleRank” that trusts those who have a large number of accepted submissions more. Over time, those with high PeopleRank may have their recommendations added immediately with after-the-fact review.

Jason doesn’t expect to use this model to build new search pages, but he does think it can improve quality and freshness of existing pages by around 20%. After all, he says, it takes 30 seconds to review an incoming link vs. the 10-20 minutes it would take to find it.

He is being very clear that he’s not competing with social networking sites like Facebook or MySpace. He points out that social networking can be either a standalone product (such as those sites) or it can be something that simply enhances an unrelated product (like Mahalo). He’s interested in the latter. He says Mahalo will have no donut throwing, zombies, or scrabble games. The social networking component, is, in fact, a bit bare. You can create a profile, become a fan of someone (when that person reciprocates, you become a friend), and see the recommendations of your friends and those you are a fan of. There’s no messaging feature, although it sounds like that may be coming. Each search result page does have a discussion board that has been made more visible and enables you to discuss the topic with others and Mahalo editors.

Jason points to this discussion forum as one way Mahalo is responsive to site owners. The major search engines are too large to scalably handle each webmaster’s concerns. But Mahalo can respond to individual questions, such as explain why a particular site didn’t make the cut on a page.

Why should you create a Mahalo profile and start recommending links? I suppose that question could be asked of sites like Digg and StumbleUpon also. Fame and recognition are certainly draws. If you’re a webmaster, you may want to get your own site’s pages in front of Mahalo editors. (In that way, this new recommendation system is more like a DMOZ submission than social networking.) Jason has left the door open for the possibility of payment or prizes for top submitters as well.

Will this improve Mahalo search results? Well, they’re not really, strictly speaking, search results. They’re hand-crafted content, and certainly more people recommending links could improve comprehensiveness and freshness. That could also make things cluttered, and reviewing recommendations could overtake the time of editors who might otherwise be creating more pages. The idea of layering social behavior on top of search could be the key to that next step of search, and while this implementation may not be that key, it’s an interesting foray into the possibility. Better would be taking advantage of user behavior that didn’t exist solely in the vacuum of improving the search experience. For instance, Yahoo! could take advantage of the behavior of Delicious users and add what they are sharing, bookmarking, and tagging as signals in their search results. And of course, using social networking data for search has been tried before. Eurekster comes to mind.

On the other hand, at yesterday’s Search 2010 webinar, held by Enquiro Research, all of the major search engines felt that the intersection of the social graph with search was valuable, but was challenging enough that tackling it was still a ways off. Marissa Mayer commented that searchers want their history to be private and manual tagging is just too difficult. Jakob Nielson felt that social networks are still too small to make an impact on something as large as search. To give Yahoo! some credit, they did point out that they are investing in social media products and are very interested in this intersection, although it sounded like their initial work will be in personalization, rather than in radically changed overall search results. But overall (despite the promise of OpenSocial), I didn’t get the sense that the major search engines are actively working on integrating the social graph with search in the way that Mahalo seems eager to do.

Mahalo is announcing a few other changes today as well. They are changing the restrictions on their content from all rights reserved to a limited creative commons license and making all content available via RSS.

Those who want to subscribe to updates or do mashups can use the OPML file to slice and dice however they choose (within the licensing guidelines).