Mahalo Answers is the newest entry into the crowded Q&A reference site space, but it offers a twist that its biggest competitors don’t: the chance to earn money by contributing to the service. Mahalo founder Jason Calacanis says the new service is the third and final piece of his original vision for Mahalo, making it a site that combines search, content, and knowledge exchanges.

Yahoo Answers is the 800-lb. gorilla in this field, with some astonishing numbers reported earlier this year: 135 million users and 500 million answers worldwide, and growing at a rate of 11 million new answers per month just in the U.S.

What Calacanis hopes will set Mahalo apart is cash. Users asking a question can offer money as an incentive to attract good answers, and after the fact, other readers can also tip the best answer(er).

Mahalo Answers also brings money into the Q&A equation by giving experts the opportunity to charge money for direct questions. An SEO expert, for example, could set up shop in Mahalo Answers and charge a small sum to anyone who wants to ask a direct question.

Money was part of the now-defunct Google Answers ecosystem, which functioned very similarly to what Mahalo is doing now. The primary differences are that Google Answers required a cash payment, while it’s optional on Mahalo, and questions on Google Answers were asked to pre-screened experts, while anyone on Mahalo can answer a question.

Yahoo Answers also has an open approach with any community member being able to answer questions. But in his blog post about today’s launch, Calacanis takes a swipe at Yahoo Answers’ free-for-all approach:

“…the chance of making some money in Mahalo Answers is obviously a lot better than the guarantee of making none in Yahoo! Answers–not that I’m comparing the two products.”

But everyone else is, as evidenced by the discussion on Techmeme.

Previously I’ve covered what I dubbed Search 3.0, how search engines have evolved toward blending vertical or specialized results into “regular” web listings. Today, the step beyond that: Search 4.0, how personal, social and human-edited data can be used to refine search results.

The Search Evolution So Far

Before going ahead, let me summarize what I covered in my past article, in terms of how search engines have changed over time to create and rank the results you get when doing a search:

• Search 1.0 (1996): Pages ranked using “on-the-page” criteria
• Search 2.0 (1998): Pages ranked using “off-the-page” criteria
• Search 3.0 (2007): Vertical search results blended into regular search results

The evolution above is not perfect. For one thing, some “Search 3.0″ blending started to happen years before 2007. It’s just that in 2007, I felt all the major search engines made the leap into Search 3.0 in a significant way.

As for Search 2.0, looking at off-the-page criteria such as links, Google kickstarted that heavily in 1998. However, some link analysis happened before then, and all the major search engines probably didn’t get on board to using it more fully until 1999-2001. But the launch of Google in 1998 remains the benchmark year in my mind, for that particular change.

The evolution is also only applicable to crawler-based search engines, those that use automation to gather web pages, store copies of them and search through the compiled index to create listings for searches. Yahoo was a major player using human power before 1996 and continued this way for years. Indeed in 1999, a majority of major search engines were presenting human-powered results. This quickly changed as Google grew. Yahoo made its human results “secondary” to crawler-based ones (then provided by Google) in October 2002. Today, all the major US-based search engines depend on crawler-based results.

To cap off the caveats, the evolution above is not the only way search engines can evolve. That’s just how things have largely gone with US-based search engines, which in turn tend to also be the major search engines for most countries around the world. There are exceptions. For example, Naver is the dominant search engine in Korea — and there, listings are largely human generated.

Search 4.0: The Human Factor

Onward to Search 4.0! As I said in my opening, to me this is the move for search engines to make use of human data as part of their ranking systems. In particular, it means human data generated by you, by those you know or by human editors.

Search engines already make use of some human data. All the major search engines, for example, monitor what we click on within the search results. This helps them determine if a particular listing is drawing more or less clicks than would be expected for the position it holds. For example, if the number two listing for a particular query is getting less clicks than “normal” for a listing in that spot, perhaps it’s a bad quality listing that should be replaced with another.

Another example: all the major search engines make heavy use of link data — and that link data is largely human data, humans both “voting” with their links and “tagging” pages by the words they use in the links. Google Now Reporting Anchor Text Phrases and Google Kills Bush’s Miserable Failure Search & Other Google Bombs provide more about how links are used in this fashion.

When I talk about putting human data into search results as part of Search 4.0, I mean things that are more aggressive or active than what I’ve covered above. I’ll start off with the most refined Search 4.0 implementation out there, Google’s personalized results.

Google: Search 4.0 Gets Personal

With Google Personalized Search, the web pages you visit, bookmark and things you click on within search results at Google are used to custom-tailor search results for you. The personalization is not as dramatic as with a place like Amazon, where if you purchase a book once, Amazon seems to continually push similar books like that at you forever. Shifts are far more subtle, mainly to help elevate results from sites you frequently visit.

To understand more, these articles go into depth about the process:

• Google Ramps Up Personalized Search
• Google Search History Expands, Becomes Web History

I’m fairly bullish on personalized search as an important addition to other factors (Search 1.0-3.0) in improving results. For one thing — better or worse — people often judge the relevancy of search results based on ego searches. Does a search engine find your home page, blog and related material when you search for yourself? Does it find your company? Personalized search is an ego search reinforcer. Because you go to your own places on the web often, Google senses that you want them to show up higher in search results, and they do. It’s a genius way to ensure anyone reviewing the service comes away pleased!

Of course, fulfilling ego searches can also be an relevancy advancement, not just a marketing ploy. There’s an excellent chance you’d have better searches if sites you visit more often get a bump in the search results. Personalized search can do this. In addition, over time, personalized search can potentially figure out other sites that are similar to those you visit and give them a relevancy boost.

Since Google expanded personalized search last year, there’s been one further major development. Personalized search uses searches over time to refine results. However, Google also has a system it is testing to refine results based on the last query you did, even if you aren’t taking part in the personalized search program.

“Previous Query” Refinement Coming To Hit Google Results explains more about how this works. It’s been used to improve the ads shown on Google for almost a year now, and it’s currently being tested to refine regular results. Google said that previous query refinement has been one of the strongest signals on how to personalized results so far.

Social Search: Promise Or Hype?

Last year, blogger Robert Scoble kicked off a round of “Facebook’s gonna kill Google” with a series of videos suggesting that because Facebook knows who your friends are, they’ll be able to apply that “social graph” data to improving search results.

The Promise & Reality Of Mixing The Social Graph With Search Engines was my response, a bucket of cold water explaining that using social data wasn’t some new idea that had never been tried before. The article went into depth explaining how Eurekster and Yahoo both assumed search could be “socialized” similar to photo sharing or bookmarking, only to find that wasn’t the case.

Yahoo had little take-up of its social search product. I’ve never seen the company explain why. My own suspicion is that take-up was low because search is NOT a social activity. I believe people tend to search when they have an immediate desire that needs fulfilling, and taking time away from the search activity to “share” with others is a distraction. Consider the person who has a broken water pipe. They might search quickly to find a plumber. They aren’t likely thinking at that moment that they want to tag and classify the search they conducted, much less the plumber they called. They just want the pipe fixed!

Eurekster has said that it found social activity worked better when people organized to build what it calls “Swickis,” search engines that hit only a custom collection of web sites related to a particular topic. Earlier this year, Eurekster formally came out of beta. However, the service has been entirely off-line for almost a week now. Practically no one has noticed, which speaks volumes to its usage and that aspect of the social search potential. Twitter, which some still view as a niche service, can hiccup for an hour and produce reams of blog attention. Eurekster goes silent, and the web stays silent about it.

I’m A Facebookholic & I Have 5,000 Friends

Still, couldn’t Facebook have more luck? For the record, when I spoke with Facebook director of engineering Aditya Agarwal about social search ideas last December, he was far more realistic than outsiders who hype what Facebook could do. In particular, he wasn’t certain how useful the social data actually would be for refining web search.

I plan to do a future article with Agarwal to explore this more. As a reminder, Facebook right now has no web search feature at all. And while it does have an ad deal with Microsoft, our previous Microsoft’s Facebook Ad Deal Doesn’t Include Search article covers how a search partner hasn’t been selected.

Let’s assume that Facebook does select a search partner, which it will need, since trying to index billions of pages and serve millions of queries each day is not an easy task (just ask Microsoft what it’s like to build that from scratch). What could it do with social data?

For one thing, it could monitor what people are clicking on in a potentially more “trusted” environment. Anyone can use web search anonymously, even sending in clickbots to make it seem like some particular listing is super hot. Having to register to be in Facebook and search from within there might make the clickstream data less noisy. But then again, it’s still a fairly open door that someone can walk through, if they want.

Facebook could tailor results based on what friends are searching on. If it knows what you and your 25 friends all seem to select from results, it could ensure those sites get ranking boosts for future searches. That’s very similar to personalized search, except it sounds full of extra friend-goodness, right?

The flaw here is plenty of people have friends on Facebook they don’t know. Some people collect friends for fun (and profit). Some people get friended by others just looking to build up their profiles. Some people you might friend not because you like them but because it’s easier to friend them than say no. Any of these instances can cause “pollution” of the social data that supposedly was going to improve your search results.

Consider also the case of someone who might work at some very conservative company but outside of work is a freeliving, devil-take-all person. Do they want coworkers who are friends to flavor their search results or those friends they hang out with when work is over?

Finally, privacy is an overlooked issue when it comes to social search. People often search for intensely private, personal things using search engines. Search engines are almost like confessionals, where people seek solutions to problems they might not tell real people that are close to them. With social search, do they have to remember to turn off a sharing feature that might be activated by default? And if it’s not on by default, will it get any take-up at all?

In the end, I think there is some potential to tapping into a social network and applying it to search. However, I still remains uncertain how that will unfold. It especially remains uncertain that this is somehow the secret sauce for anyone to jump past the current state of search.

Return To Humans: Hello Mahalo!

Earlier, I’d mentioned how Yahoo started off using human beings to create its search listings in the days before Google existed. Over time, the human soul in search was lost to reliance on the supposed scalability of machines. Anyone who wants to see how much we’ve handed over to machines need only search for buy cialis online on Google. At the moment, the results are littered with online discussion forums that have been “borrowed” by affiliates and others hawking deals.

Those pages will sit there for a day or two or three or potentially weeks, as Google usually tries to find an algorithmic solution to getting rid of them. The idea is you might have to suffer a bit in the short term until a long-term cure is found. But then like a virus that mutates, something else gets through, requiring a new long-term cure.

Enter humans. A human editor, reviewing results like that, can immediately spot junk that should get yanked. Even better, a human editor could act as a curator. How hard can it be to find 10 quality sites that should come up for that or other terms?

That exact human solution, of course, is what Mahalo has been banking on. Mahalo, launched last year, uses human editors to hand-pick top results. For background on the service, check out these past articles:

• Mahalo Launches With Human-Crafted Search Results
• Mahalo Greenhouse: Get Paid For Writing Search Results
• Mahalo Follow: Toolbar Gives You Human-Powered Alternatives To Searching, Surfing
• Mahalo Adds The Social Graph To Search
• Mahalo Adds More Social Features

As part of a talk I do on Search 3.0 and Search 4.0, I have some screenshots from last year that illustrate well how a human can indeed do better than the machines, for some queries. Remember the fires in Southern California at the end of last year. After a series of wide ranging ones, Malibu was hit with a second one a month later. Here’s what those searching on Google got in response:

The news box at the top is great, but sometimes searchers skip past things like this and go to the first “real” result. That’s a story about the Malibu fire early in 2007, not at the end of the year. Other results were largely about the fire of October 2007, rather than November 2007 (which is what many searchers at the time I snapped this would have been interested in).

Here’s Yahoo:

Again, news results at the top, then unlike Google, places you’d expect to find news about the fire — the local paper; ironically a map of the fires on Google Maps that Google itself didn’t return; the Malibu city web site, as well as the fire department.

Here’s Microsoft Live Search:

Very similar to Yahoo — a news box, the fire department, the Red Cross. What’s not to like? Well, let’s look at Mahalo:

Note at the top that Mahalo’s human editors understand there’s a different fire that happened in the past, in October 2007, and offer a link to a page about that. Then there’s a nice list of news sources, followed by coverage by date. Over to the side, a synopsis of the current situation. If you could see more of the page, there was lots of other categorized information.

It’s nicely done. It’s very helpful. And it was created with a human thinking about what other humans might want to see, rather than machines guessing.

Scaling Humans

So is Mahalo founder Jason Calacanis onto the Google-killer, human crafted results? No. I think human review can be part of the solution, part of the Search 4.0 addition to what we have out there already — but humans can’t craft pages for every possible search. In addition, it’s hard to keep those pages maintained once they’ve been made. It’s also easy to cross over from being a search resource that points to other resources to becoming instead a destination site. I think a good search engine avoids that (and Who’s Ranking For Knol? Hello, Wikipedia! has more on this topic).

Mahalo can also be overwhelming. Try a search for hillary clinton and there’s category after category. Background links. News links. Photos. Videos. Bio links. Blogs and message boards. Plus, there’s even more. I think at some point, you want your search engine to make some key choices for you, not flood you with so many that you don’t know where to begin.

Another issue is that what Mahalo’s human editors do, machines can get close to. Hakia especially stands out here. Search for hillary clinton there, and you’ll see how listings are grouped into categories like Awards and Biography without humans being involved (and see Social Networking Through Search: Hakia Helps You Meet Others for background on how Hakia works).

More Humans

There is another major search project involving humans: Search Wikia. Backed by Wikipedia cofounder Jimmy Wales, the service aims to involve humans in rating pages, annotating them and helping determine the ranking algorithm for choices the machine side of the project makes.

The articles below have more background on the service:

• Search Wikia: Not Even A Remote Threat To Google
• Search Wikia Adds Alpha 0.2 Features & More

Right now, the quality of the service is poor, as Search Wikia itself readily admits. There’s still lots of work to be done — and even with it, it might never succeed. But allowing humans into the process is, in my view, a good thing.

Indeed, even Google understands this. Last year, Google started doing some education about how human “signals” are already incorporated into its algorithm (see Google’s Human Touch and Google & Human Quality Reviews: Old News Returns). Aside from this, last year it also started testing a way for people to annotate search results — add those they like, remove some, suggest other ones.

Watch Personalized Search

Overall, there’s a role for humans, a way for them to be in the search process to enhance results. Actually, there will be several ways for them to be involved. Exactly how remains to be seen, of course.

Of the things I’ve outlined — personalized search, social search, human editors — I think personalized search is the one that will emerge as the major part of Search 4.0. That’s not to discount other things being tried, and they’ll contribute in some ways. But to me, personalized search has the most potential for another big relevancy leap. We’ll see!

At our SMX Social Media Marketing conference this week, we had a great panel on the future of human powered search. Jason Calacanis of Mahalo, Jimmy Wales of Wikia Search, and Steven Marder of Eurekster all took part. Jason had some remarks on SEO that set off the usual wave of upset. But as I commented to those who weren’t at the panel, by the end of Q&A, Jason – along with Jimmy Wales and Seven Marder — were agreeing about the usefulness of SEO. It’s all down to the definitions.

Below you can hear Jason’s presentation yourself, then you can hear the Q&A portion that covered search marketing and human powered search. Note that the video production could be better. Hey, I just got a Macbook — it’s my first time playing with what it can supposedly do. Do don’t hassle me over the titles that could be better. Also, I will get the entire session up with the presentations from Jimmy and Steven, along with the further Q&A. But first, Jason:

The Q&A:

And here’s some of the reaction to initial reports of his comments, before anyone not at the show could hear what he said in full:

• My Plea to SE Conferences: Turn Calacanis Away, Eric Lander
• My Plea to SE Conferences: Turn Calacanis Away, Sphinn discussion on post above
• Why Calacanis Should Not Be Allowed to Speak at Conferences, Slightly Shady SEO
• Why Calacanis Should Not Be Allowed to Speak at Conferences, Sphinn discussion on post above
• Jason Calacanis has pissed off SEO’s… again. Seriously., Sphinn
• Srsly are SEOs Really That Easy To Manipulate?, ShoeMoney

And here’s further coverage of the show (for initial coverage from the first day, see SMX Social: Mahalo To Do Microformats, Search Wikia Adds Alpha 0.2 Features & More):

• 20 Take-Aways from SMX Social Media – Scott Clark, Scott Clark
• Evangelist – The Marketer’s Role in SMM, BruceClay.com
• Micro Communities, BruceClay.com
• Wikipedia, Yahoo Answers & Answer Sharing, BruceClay.com
• Social Media Expo (SMX) Report from Long Beach, California, Wpromoter
• SMX Social Media Conference in Long Beach, MindCradle
• Rand Fishkin | Jason Calacanis | SMX Social, Gary Pool
• SMX Social Media Coverage Round Up, BruceClay.com
• SMX Social Media Pics, Scott Clark @ Flickr

News from the first day of our SMX Social Media Marketing conference. Mahalo is now doing microformats as a way to enhance its search results and allow local businesses to be added to your address book. Wikia Search has gone to "Alpha 0.2" with new features. And other conference coverage, below.

Jason Calacanis of Mahalo, Jimmy Wales of Wikia Search, and Steven Marder of Eurekster all spoke at the end of the day on our Social Search: The Human Challengers panel. Lisa Barone provides live blogging coverage in her Keynote – Social Search: The Human Challengers post at the Bruce Clay blog.

During that session, Jason shared that Mahalo is providing microformat support. Mahalo Adds Microformats from Mahalo’s Sean Percival covers how this works, complete with screenshots. Check it out! See also further discussion on Techmeme.

Also during the session, Jimmy Wales shared that Wikia Search has added several new features as part of a new alpha release, one he dubbed "Alpha 0.2." Wikia Search doesn’t seem to have news up yet on its own site, but Wikia Search Launches Major Enhancements to Search Alpha from CenterNetworks covers how you can do things like preview pages and copy sections of them into search results. You can also add related search terms to queries, add pages you think should be added to a search result, and delete material you think doesn’t belong.

Scary, freaky? You bet — but Jimmy also covered how, like Wikipedia, the history of all this can be seen, which potentially provides some self-policing. Try the new features on the test site here. And further discussion can be found on Techmeme.

For other session coverage from the conference, see:

• Social Media Marketing Essentials, BruceClay.com
• Linkbait – Chumming for Traffic on Social Media Sites, BruceClay.com
• Extra! Extra! The Social News Sites, BruceClay.com
• A Marketer’s Guide to Social Bookmarking, BruceClay.com

We’ll also do a Day 2 post tomorrow with further coverage and add to it anything we’ve missed. Also, you can keep up with live blogging both at the Bruce Clay blog and via Twitter. Yep, Twitter. Michael Gray is diligently microblogging on our SMX twitter account.

Mahalo has just launched several new features that entwine the human-powered search engine even further into social networking and online community building. The Mahalo Follow toolbar now enables you to post links to Delicious, Ma.gnolia, Mahalo Social, and Twitter with the click of one button. The sidebar now displays quick tips when you’re on sites like Twitter and Gmail.

In addition, Mahalo now lets you create “stub” pages (much like you can on Wikipedia). Jason Calacanis assures us on his blog that he has 50 full time people watching user-contributed content for spam and will not only remove spam, but might ban the domain and user as well.

For the purposes of this article, I decided to try the Mahalo Follow toolbar. I don’t generally use Delicious, and I’ve never used Ma.gnolia, so the cross-posting features aren’t as useful to me as they may be to others. If I could add the services I do use, it might be more interesting. I also didn’t love the fact that I had to give Mahalo my Delicious and Twitter login information.

The sidebar tips are potentially useful (at least until you have the ones you need memorized), but the sidebar itself takes up a lot of space. I also recently installed the Remember the Milk add on for Gmail, which shows up as a right sidebar, and I think this may be causing the Mahalo sidebar to have a funky layout in Gmail.

I also admit that I don’t get the sidebar in general. The help page says that it shows Mahalo content that’s related to the page you’re viewing. However, when I was viewing the Ma.gnolia page, the sidebar showed me how to make a dirty martini (perhaps because it picked up the word “mixed” in the “mixed martial arts” heading on the Ma.gnolia page. When I view my blog, it shows me content about Twitter, Biz Stone, and batteries, and I suppose that looking at my recent blog posts, that’s more relevant than anything else, but still not necessarily super useful.

As with much of Mahalo content, the best results seem to be that which are hand-picked by human editors, such as the new sidebar tips.

It remains to be seen how popular the new stub pages will be. Wikipedia has proven that users can get passionately involved in creating this type of content, and in the case of Mahalo, the pages will be overseen by Mahalo-employed editors, rather than volunteers, which may increase accuracy and objectivity. It’s not clear what advantage these pages have over existing sites such as Wikipedia, though (both in terms of what would motivate people to create them and to use them).

Rich Skrenta — who, aside from creating the first computer virus, is more notable to search as a cofounder of the Open Directory Project and the Topix news search engine — has announced he’s founded a search start-up. A stealth one, as TechCrunch puts it. Don’t we already have several stealth search start-ups? Yep. Here’s a guide to who’s who.

Blekko

What we know so far about Blekko isn’t much, and TechCrunch has the most details in its The Next Google Search Challenger: Blekko post from yesterday. Apparently Rich founded the company in September 2006, along with five other former Topix employees, after he left Topix in June.

Rich told TechCrunch not to likely expect anything public until 2009. I agree with Michael Arrington at TechCrunch that Rich has a track record that makes him well worth watching. The Open Directory was an initial success, though the model didn’t scale well. Some of that was within the founders’ control but had more to do with AOL’s lack of backing. The company should be dragged into the International Court Of Search Crimes and be forced to sell the ODP to someone who will support it properly. Topix has built a reputation and is still standing and succeeding — though I’d say it still has far to go to seriously threaten Google or Yahoo.

Rich adds a bit more in his Why Search? post today:

Having just spent 5 years in the media space, I’ve come away with the idea that editorial differentiation is possible. But the editorial voice of a search engine is in the index…so it has to be algorithmic editorial differentiation.

So far, it doesn’t sound like a social networking play like some of the others. We’ll be watching, Rich. Also see discussion today on Techmeme.

Powerset

Powerset is now a classic example of why you WANT to be a stealth start-up and say little. That’s because when you get too much early press — in part through your own doing — then fail to deliver anything, the hype can swing back at you hard.

The company came to light back in October 2006 via VentureBeat, with the twist being that natural language search would be the way forward. That caused me to write a long rant about the hype of natural language search in reaction. From the top of that:

This is a rant. It’s a rant from over 10 years of watching people trot out natural language search as the "killer" solution to the current state of search, something that’s happening once again with Powerset. That’s a search engine you can’t even use at the moment, but the hype will no doubt continue. To counteract that, my thoughts on and some history about natural language search.

Natural language search makes a compelling pitch for those who really don’t know search or haven’t heard the natural language mantra before. I’ve seen the pitch time and time again. You:

• Pick out an example that shows how "bad" search is on an existing search engine
• Demonstrate how natural language search would work better on your service
• Sit back and collect the press attention

I then went on to detail how natural language search had been hyped and tried over the years. The short story is this: It doesn’t take much natural language analysis to figure out what someone wants when they type in "britney spears nude" or "hotmail." In addition, by and large I don’t believe enough people will change their basic search habits to enter long sentences when searching any time soon.

Since that time, we’ve pretty much had nothing out of Powerset other than the launch of Powerset Labs in September 2007. That launch hasn’t produced any cool applications that I’ve seen or heard about, nor much buzz. Instead, in November, we got a management shake-up.

For a more formal chronicle of the company’s developments, check out this area at VentureBeat and these search results at TechCrunch.

Finally, while I’m harsh above on Powerset, I actually had a long visit with the company in the middle of last year and was deeply impressed with the effort going on there. I’m still working on a long write-up to explain what’s happening. But in a nutshell, Powerset is trying to literally comprehend or understand each page on the web.

Today’s search engines don’t know what a page is about by reading words. They’re more or less doing pattern matching — finding pages that contain words similar to what you search for (or pages relevant to those words based on linkage). Powerset literally is trying to read and understand what a page is about the way a human reads a page and knows it is on various subjects.

I don’t see that as making it a better search engine that Google. Instead, I think it may eventually give it the ability to create a unique "auto-Wikipedia" style site, assembling knowledge pages on any subject automatically. I also think that there will eventually be some search benefit in comprehension of pages, but exactly how that will play out I suspect is part of being with an existing search engine and a more traditional model. With the array of patents Powerset has lined up, I suspect it will eventually get acquired by Google, Yahoo, or Microsoft rather than rollout its own product. But we’ll see.

Hakia

Like Powerset, Hakia has played the natural language search game. Unlike Powerset, it has a product anyone can use — live since at least the middle of 2006.

Again, I’ve been working on a long write-up on the inner workings of Hakia and have yet to finish it. It’s complicated, and I mainly want to cover what I find to be the real use of their technology — the ability to create custom "gallery" pages and understand those are related to particular searches.

It’s easier to show you what’s impressive. Search for hillary clinton, and you get a nice page showing news, her official site, biography pages, blogs & fan sites, news & interviews, and more. It’s very Mahalo-like, except it doesn’t require human editors like Mahalo and predates Mahalo by a year.

That categorization is something I know the major search engines could do, if they wanted. So far, they don’t. And so far, despite Hakia talking about its rising traffic, it has yet to make a serious mark. Moreover, in October, it made a serious shift to allow social interaction with its results. That’s a sign that the original plan that "natural language will win all" has failed to do so; therefore, another twist is needed.

Social Networking Through Search: Hakia Helps You Meet Others from Vanessa Fox here at Search Engine Land covers the change, plus it gets into the natural language indexing stuff I mentioned earlier that makes Hakia unique, plus has examples of gallery pages.

Mahalo

Credit to Jason Calacanis. He said he wanted to take on Google, then wasted no time getting Mahalo rolled out. OK, he also says he’s not taking on Google — just focusing on the top searches that he thinks would be better with human review. Sure, you aren’t taking on Google, Jason.

To date, Jason reports that Mahalo’s traffic is growing and strong. But to date, I’ve certainly see no webmasters taking about what a traffic driver Mahalo is. It would be early to call it a raging success, but it’s a nice alternative to have. Indeed, later this month I’ll finally finish my Search 4.0 piece that picks up from the conclusion of my Search 3.0: The Blended & Vertical Search Revolution article last November. I’ll show some examples of how the human element at Mahalo can and has kicked some Google and traditional search engine butt — though also how it isn’t the panacea some expect.

Some of our past coverage of Mahalo:

• Mahalo Launches With Human-Crafted Search Results
• Mahalo Greenhouse: Get Paid For Writing Search Results
• Search Spam Fight - Mahalo: 1; Squidoo: 0
• Mahalo Follow: Toolbar Gives You Human-Powered Alternatives To Searching, Surfing
• The Promise & Reality Of Mixing The Social Graph With Search Engines
• Mahalo Adds The Social Graph To Search

Search Wikia / Wikia Search

Wikipedia founder (as he prefers to be called; Wikipedia itself calls him cofounder) Jimmy Wales made waves a year ago when he said he’d take on "closed" Google with humans and a transparent search engine. Called Search Wikia (but, confusingly, it’s also called Wikia Search), Wales has grabbed attention from the press over the past year. Slamming at Google as a scary closed thing gets you good mileage, especially when you helped establish Wikipedia, a threat Google takes so seriously that it may launch its own Wikipedia-style site, Google Knol.

Now Wikia Search is at hand. A private "pre-alpha" test started in late December, an invite-only thing I still find odd for a service that’s supposedly all about the "transparency." But on Monday, the general public will finally get a look at whatever Wales and his team have concocted. In the meantime, while Wales still hasn’t posted any news since July 27 to the "news" section of Search Wikia, press reports tell us so far:

• Only a tiny 50 to 100 million pages will be indexed at launch. The major search engines today have tens of billions of pages indexed. (AP)
   
• There will be a high degree of human editorial influence, though whether that’s over the algorithm or the search results on a per-query basis remains to be seen (CMP)
   
• An early screenshot suggested that Search Wikia might be evolving more into a Facebook-style service, perhaps with some ways for users to share results (Matthew Buckland & Wired)

Some of our past coverage of Search Wikia:

• Q&A With Jimmy Wales On Search Wikia
• Search Wikia Takes Steps To Crawl; Acquires Grub
• Search Wikia Gets Open Source Categorization Software
• Search Wikia Launches In 2007 With Private Beta

Cuill

Arguably the stealthiest of the stealth start-ups, Cuill (pronounced "cool") has an impressive pedigree with its three founders: Tom Costello of IBM’s WebFountain project and Anna Patterson and Russell Power of Google’s TeraGoogle project, its massive search index. And last year, former AltaVista founder Louis Monier — who later went to eBay as its first eBay Fellow, then to Google — jumped ship from Google to join Cuill.

I talked with Cuill earlier this year to understand a bit more about what they are doing, but the details are still being held very closely. The main difference between Cuill and everyone else I’ve named above is that Cuill is founded by people who understand and have dealt with firsthand the challenge of indexing billions of documents.

Cuill recently took on more funding. Louis is also going to be doing a keynote at our SMX West search marketing conference, held in Santa Clara, California from Feb. 26-28. I’m thrilled to be having him since there are only a handful of people who have worked for the "old" Google (AltaVista), the current Google (when he was at the Big G), and a potential future Google (Cuill).

And The Winner Is…

If you think the future of search is on smart automation, Cuill’s definitely one to watch, and perhaps Blekko as well. If you think it’s the growth of humans, Mahalo and Search Wikia are your better candidates. The reality is that success will likely be a blend of the two. For the human services, a real open source index would be a big help — see Google: As Open As It Wants To Be (i.e., When It’s Convenient) for more about this.

But the reality is that all of these services will have an incredibly tough time to beat Google.

Google came along at a very special time, as I’ve long written. It had better technology at a time when all the search engines had abandoned improving search, since that was seen as a loss leader. The money was in portal features.

Today, search is a multi-billion dollar industry. If someone with a serious search threat comes along, you buy them (such as with YouTube), or you start to develop your own rival if it seems a real threat. Google’s not omnipotent — but you’ve already got a space where it’s Google, Yahoo, Microsoft, and Ask all seriously fighting it out (and the latter three, despite their funding and experience, still struggle against Google as being synonymous as a trusted search brand for most users).

To date, Google is the real exception of "a better mousetrap wins." It’s far more likely the companies above, if they do gain traction, will end up being purchased for a large amount by one of the existing "search utility companies."

Ah, sweet irony. Yesterday I spent some time raising concerns about knowledge aggregation sites like Wikipedia and the forthcoming Google Knol potentially ranking tops for every search conducted. Today, what’s in the top results for Knol? Yep — a new Wikipedia page on the topic!

The page was created yesterday and took less than 24 hours to show up. Looking at the top results for Knol is also fascinating in how until last week, the Google project wasn’t announced, so the results had no reflection of it. Today, they dominate the page:

Let’s run down the list.

1. KNOL is also the ticker symbol for Knology, and the Yahoo Finance page about that company has managed to hang on to the top spot.
    
2. The official Google Blog post on Knol comes next.
    
3. The official Google screenshot of an example Knol page shows up third. Some SEO advice to Google: Get a title tag on that page so it doesn’t look all weird when listed. You might also want to put a link at the top of the page over to your blog post so people hitting the screenshot have somewhere to go for more information.
    
4. Search authority Tim Bray warms my heart by covering how "transparent" Wikipedia actually is pretty closed given "a forest of acronym" and other issues that make me nod my head in violent agreement. But he doesn’t see Knol as a solution.
    
5. MarketWatch’s page about Knology comes next.
    
6. Mashable’s write-up on Knol is fifth. Sniff. We were one of the few places pre-briefed by Google on Knol and had an article with details not in the official blog post, which is what the Mashable article and virtually all other news stories were based on. But we get relegated to position 20 in the search results. Sniff. But congrats to Mashable, and we’ll look forward to when Search Engine Land is a bit older. With age comes authority and an easier way to make it to the top. We only just turned one!
    
7. Wikipedia makes it at sixth. It’ll be interesting to see if the page rises over time. FYI, Squidoo has three pages about Knol now. I like this one that’s just a copy of the official Google Blog post. I guess the author missed the Google copyright statement at the bottom of the post. The other two (here and here) are pretty basic. Mahalo’s got a nice page of mainly news commentary (though our write-up, sniff, isn’t listed). As for Yahoo Answers, three questions: how can someone get a Knol invite (here, and you can’t); are there reasons for Yahoo Answers folks to be afraid of Knol (here), and how does Knol compare to Wikipedia (here).
    
8. Noah Brier’s two paragraph summary of Knol pulls off a nice coup by getting into the top ten.
    
9. News.com’s write-up on Knol.
    
10. Wired’s write-up on Knol.

Also, I took a quick spin at Yahoo, Microsoft, and Ask. Main differences?

• Yahoo results are pretty similar to Google, though the Dutch Knol-Online makes it in the top results.
   
• Microsoft gets Knol Computers, Knol Farms, and Wikipedia’s Knol Tate page into the top results. Let’s hear it for diversity in search listings! You also get finance pages about Knology. As for Google Knol, you get one single page of ZDNet coverage — not even the official Google Blog post. Come on, Microsoft – that post should be there.
   
• Ask has even more diversity, from Ryan Knol Designs to this page that mentions someone named Knol, among many other people. Google Knol is covered by only a single News.com article about the service. The official blog post doesn’t show, and that’s just as disappointing as with Microsoft.

Everyone’s been talking about how the social graph is the next evolution of search. Search 4.0. The next step forward after Search 3.0’s blended and personalized search. Today, Mahalo is taking that next step and adding a social layer to their search results. Jason Calacanis, Mahalo founder, says that the problem of search will be solved by a combination of machines, human curation, and social interaction, and with today’s launch of Mahalo Social, Mahalo adds the beginning of that elusive social interaction. The new features include profiles and the ability to recommend links for search terms. Much like Digg or Delicious, users can add friends and see what those friends are recommending. Below, more information on how Mahalo plans to deal with spam, work with webmasters, and if this approach will scale.

Jason acknowledges that Mahalo is a content site rather than a search engine. 30% of the site is editorially generated by their staff, but it’s presented in a search context. Mahalo contains about 26,000 pages so far and Jason says they’ll never cover the long tail of search. But he does think they’ll catch up with the amount of content on Wikipedia, and within five years should be covering up to half of what people are searching for. (Don’t worry Jason, it’s mostly Britney Spears and sex. And I see you’ve got those two subjects covered.)

Mahalo employs 60 full-time editors, 400 paid contributors, and 3,000 “Greenhouse” volunteers. This enables them to add around 1,000 new pages a week, but they need a way to find more quality links and gain more insight into what people want pages about. So, they’re now handing Tom Sawyer-like paint brushes out to the rest of us and looking to crowdsourcing to help paint the search results fence. Harnessing the wisdom (or lack thereof, as the case may be) of crowds helps them overcome what they feel is currently their greatest hurdle: adding and updating pages.

Anyone can now create a profile on Mahalo, then start recommending links for search terms (regardless of whether those terms have existing Mahalo pages). You can submit using either the Mahalo toolbar or directly from the site.

How will Mahalo combat spam? Each link will be reviewed before it’s accepted. You can see your accepted/rejected stats on your profile page. Accepted links appear in the “user recommended links” section of the search page, and anyone who contributes links will be listed in the “contributors to this page” section.

Mahalo won’t use a Digg-like system to enable people to vote on links. Jason says he’s looking for quality over quantity and doesn’t want users to simply skim a list of headlines and vote for what catches their attention. Instead, votes accumulate based on multiple recommendations.

All recommendations go to human editors, who either approve them, wait for further recommendations, or ban them. Is this scalable? Jason says he doesn’t expect a lot of participation (he cited the fact that most user-generated content tends to come from a small fraction of a community, and he expects Mahalo to be no different) and claims he’ll be happy with 50 people participating in the first month, with a few thousand by the first year.

However, he is looking at ways to scale this a bit, just in case he gets a few more than 50 signing up. For instance, they may implement a kind of “PeopleRank” that trusts those who have a large number of accepted submissions more. Over time, those with high PeopleRank may have their recommendations added immediately with after-the-fact review.

Jason doesn’t expect to use this model to build new search pages, but he does think it can improve quality and freshness of existing pages by around 20%. After all, he says, it takes 30 seconds to review an incoming link vs. the 10-20 minutes it would take to find it.

He is being very clear that he’s not competing with social networking sites like Facebook or MySpace. He points out that social networking can be either a standalone product (such as those sites) or it can be something that simply enhances an unrelated product (like Mahalo). He’s interested in the latter. He says Mahalo will have no donut throwing, zombies, or scrabble games. The social networking component, is, in fact, a bit bare. You can create a profile, become a fan of someone (when that person reciprocates, you become a friend), and see the recommendations of your friends and those you are a fan of. There’s no messaging feature, although it sounds like that may be coming. Each search result page does have a discussion board that has been made more visible and enables you to discuss the topic with others and Mahalo editors.

Jason points to this discussion forum as one way Mahalo is responsive to site owners. The major search engines are too large to scalably handle each webmaster’s concerns. But Mahalo can respond to individual questions, such as explain why a particular site didn’t make the cut on a page.

Why should you create a Mahalo profile and start recommending links? I suppose that question could be asked of sites like Digg and StumbleUpon also. Fame and recognition are certainly draws. If you’re a webmaster, you may want to get your own site’s pages in front of Mahalo editors. (In that way, this new recommendation system is more like a DMOZ submission than social networking.) Jason has left the door open for the possibility of payment or prizes for top submitters as well.

Will this improve Mahalo search results? Well, they’re not really, strictly speaking, search results. They’re hand-crafted content, and certainly more people recommending links could improve comprehensiveness and freshness. That could also make things cluttered, and reviewing recommendations could overtake the time of editors who might otherwise be creating more pages. The idea of layering social behavior on top of search could be the key to that next step of search, and while this implementation may not be that key, it’s an interesting foray into the possibility. Better would be taking advantage of user behavior that didn’t exist solely in the vacuum of improving the search experience. For instance, Yahoo! could take advantage of the behavior of Delicious users and add what they are sharing, bookmarking, and tagging as signals in their search results. And of course, using social networking data for search has been tried before. Eurekster comes to mind.

On the other hand, at yesterday’s Search 2010 webinar, held by Enquiro Research, all of the major search engines felt that the intersection of the social graph with search was valuable, but was challenging enough that tackling it was still a ways off. Marissa Mayer commented that searchers want their history to be private and manual tagging is just too difficult. Jakob Nielson felt that social networks are still too small to make an impact on something as large as search. To give Yahoo! some credit, they did point out that they are investing in social media products and are very interested in this intersection, although it sounded like their initial work will be in personalization, rather than in radically changed overall search results. But overall (despite the promise of OpenSocial), I didn’t get the sense that the major search engines are actively working on integrating the social graph with search in the way that Mahalo seems eager to do.

Mahalo is announcing a few other changes today as well. They are changing the restrictions on their content from all rights reserved to a limited creative commons license and making all content available via RSS.

Those who want to subscribe to updates or do mashups can use the OPML file to slice and dice however they choose (within the licensing guidelines).

I’m having a bad day. Aside from my desktop crashing, we get another spate of "let’s blame SEO" to start my morning off. Robert Scoble uses that theme as a launching pad for a series of videos on how Facebook potentially could be a killer search engine — regardless of the fact he seems to have no clue that "social graph" or social networking mixing has been tried and abandoned with search. Having watched his videos, which have sparked much discussion, I’ll do some debunking, some educating for those who want more history of what’s been done in the area, plus I’ll swing around to that New York Times article today that ascribes super-ranking powers to SEO. Plus, I’ll use the F-word along the way. I said it was a bad day.

Robert’s excited about "social graph search," which is the idea that if you know a network of people, you can use their connection to improve search results. It’s a "revolution" coming in search that will overtake all the major search engines, he says. Maybe, but it’s not like we haven’t heard this before. I’ll go through his arguments, but it really feels like this is more about getting attention to Robert’s videos, period.

Part 1 of Robert’s social graph video series starts off by telling us that there’s no way we’d have gotten to his videos from a search engine. That’s absurd. People write about what’s in video content all the time. Want to see the Lazy Sunday video? Oh, look — I found it number one on Google without Google needing to analyze the words inside the video.

That’s the real point that Robert’s trying to make, of course — that search engines typically don’t analyze all the words within a video in the way they read the words in a web page. Want to understand more about that? My Video Search Challenge Isn’t Speech Recognition, It’s Content Owner Management post from February explains this in more depth as well as why it really hasn’t been an issue. In particular to Robert’s argument, it’s because there are plenty of people who will reference what’s in the video content in more user friendly and search engine friendly HTML text.

The meat of his first part is to talk about three different types of search engines: crawlers (like Google), Techmeme and Mahalo to discuss how they are or are not "SEO resistant," as if SEO is a bad thing — you know, that SEO equals spam.

SEO is not spam. It’s like saying email is spam. There’s email; there’s email marketing; there’s email spam. These are all different things. You want to better understand why SEO isn’t spam? Then read the posts below:

• Yes Virginia, SEO Is Rocket Science – Defending Search Engine Optimization Once Again
• Defending SEO, Yet Again!
• Why The SEO Folks Were Mad At You, Jason
• From My Inbox: More Defense Of SEO
• SEO: Real Skills That Can Protect Your Traffic

Want to be like Robert — and Jason Calacanis — and keep equating SEO with spam? Then fuck off.

I don’t think I’ve ever used the F-word in any of my writing, and my apologies for being so crass. But I’ve had enough of people trying to advance their own personal agendas (Jason hoping someone will care about Mahalo; Robert hoping someone will watch his videos) on the back of an industry that is full of plenty of people who do good work.

Last week, I was part of a meeting at Google along with a number of notable SEOs, being asked about ways Google could be better. This group wasn’t pushing for Google to make it easier for them to spam the listings. A chief concern they had was how Google (along with other major search engines) continues to have difficulties identifying original source documents. You know — you publish your blog post, then some other site with more authority than you picks it up, and then that site gets the top ranking. SEOs are leading the charge to help site owners get a fix for this overall. But all people like Jason and Robert want to do is characterize them as evil comment spammers for their own personal gain.

Back to the video. Robert goes through how search engines make use of "on the page" factors, though he doesn’t call it that, and greatly simplifies the process. But yes, search engines look at the frequency and location of words on a page to determine if a page is relevant to that.

Robert then explains that PageRank is also used, using incorrect shorthand for link analysis that’s part of "off the page" factors that search engines use to rank pages — looking at the quality and the context of links to a page. My What Is Google PageRank? A Guide For Searchers & Webmasters post from April goes into more depth about what exactly PageRank is and how it is not the same as link analysis. Give it a read, Robert.

Next, we get the news that paid links are hard for Google to tell apart from "real" links. Actually, Robert says Google can’t tell the difference between them. In reality, it can easily identify many types of paid links. But not all of them.

Apparently, the SEO community feels it’s its "birthright" to stick paid links into pages. Actually, Robert — there’s disagreement about that within the SEO community, and the bigger audience that feels it has a birthright to do what it wants with paid links are the content owners themselves. SEOs aren’t selling the links; they’re buying the real estate that others are selling.

Robert then shifts gears to Techmeme and how, in his view, news won’t get on the site until someone starts to blog about it. Um, yeah — that’s part of the "meme" part of Techmeme. New stuff can (and does) hit the site pretty quickly, too, since important blogs catch things fast and talk about it. FYI, Q&A With Gabe Rivera, Creator Of Techmeme from me in January talks about Techmeme and how it works in more depth.

Techmeme is described as an SEO resistant site. Sure, in the sense that you’re dealing with a smaller source list. From that reason, Google News is more resistant. Any vertical or specialized search engine that deals with a subset of sites is SEO resistant (or more correctly, spam resistant).

Mahalo comes up next and how by using a small number of human editors, it can be harder to spam. Sure. So’s the Yahoo Directory. You remember the Yahoo Directory, right? It used, um, a small number of human editors to categorize the web. Advances in crawler-based search engines meant you could get really good relevancy and be spam resistant, which caused the Yahoo Directory to effectively be abandoned by Yahoo. Mahalo’s approach to custom-tailor the most popular searches is interesting — but despite heaps and heaps of publicity the new service has had showered upon it, it still hasn’t gained any real traction among searchers. Mahalo Launches With Human-Crafted Search Results from me in May describes the service in more depth.

Finally, Robert turns to Facebook, talking about the "social graph" term that’s now being bandied about as this month’s new Kool-Aid to drink. I’m being harsh — there’s obvious value in being able to look at the connections between people and form a ranking mechanism that can be applied to things. SocialRank, PeopleRank — whatever you want to call it. The idea at the moment is that Facebook especially has it, so everyone else better look out.

That leads to Part 2, where how adding the social graph to existing search technologies will really change the search game. Wow. Wow! I mean, yeah, never heard that before.

Look, Robert, back in January 2004, Eurekster launched with the promise of mixing the social graph with other search criteria, to improve results. As I wrote back then:

Personalized search? The concept has been that by knowing some things about you, a search engine might refine your results to make them more relevant. A teenager searching for music might get different matches than a senior citizen. A man looking for flowers might see different listings than a woman.

Eurekster’s twist on this concept is to provide personalized results based not on who you are but who you know. Friends, colleagues and anyone in your Eurekster social network will influence the type of results you see….

The potential of using your friends or colleagues is enormous. Imagine Eurekster being used by all the employees of a medical research firm, where many might do similar medical-related queries. With Eurekster, all the employees can be linked together and benefit from the searches and selections made by their colleagues.

Libraries are another institution that might latch on to the Eurekster concept. Librarians are constantly asked by patrons for assistance. Eurekster would allow librarians to collaborate invisibly with each other and share what they’ve found to be the best for various queries.

There are downsides. Not all of my friends have the same interests as me. In addition, as my social network grows — because my friends invite their friends and so on — commonalities that are useful get diluted.

Eurekster is still out there, but the idea of a network of friends influencing search results seems to have died at some point over the years. Viva not the revolution Robert was promising us.

Well, maybe Eurekster had no luck with that particular model since the company was small. Well, Yahoo’s not small. And in June 2005, it rolled out Yahoo My Web 2.0, which promised to bring social networks into search. As I wrote then:

After seeing what was planned, I remarked to Yahoo senior vice president of search Jeff Weiner sitting next to me that they were building "an eBay for knowledge." Jeff was already literally bouncing at times with excitement in showing the new system, and the remark made him smile even more broadly.

He smiled because that’s exactly the Yahoo goal. My Web is Yahoo’s community rating system for information. Just as you buy things on eBay depending on ratings to know if you’ll trust a seller, My Web is what Yahoo hopes will help you choose more wisely the information you receive, whether you actively check reviews, contribute or remain an ordinary searcher who completely ignores the tagging and social search components.

In short, Yahoo’s not banking on tagging — the categorization of material — as a way to help people find things better. It’s banking that the mere act of saving things at all, even without tags, will give them a clue about what are trusted pages across the web. By looking at patterns of saving, Yahoo will have trust networks to tap into….

We’ve had a generation of search engines that depended on on-the-page factors such as word location and frequency. We’ve had a current second generation that tapped into link analysis, looking at how people are linking and what they say in links.

Personal search is that third generational jump, and Yahoo’s flavor of personal search is a social network one that it hopes will improve relevancy in web wide results in the way that link analysis helped drive back spam and improve relevancy years ago.

"We’re creating personal anchor text for pages, but by having a trust network, we can actually pretty much eliminate spamming," Walther said.

Guess what? Still no revolution. The masses didn’t descend upon Yahoo My Web to form networks and save search results. In fact, Yahoo pretty much pulled back from the product, even dropping inline integration with it from regular search results back in October 2006.

But before Robert gets into applying social networks to search, he prattles on about Mahalo again being so superior to Google. Oddly for a video, he doesn’t show us any of the search results pages he’s talking about, saying at one point he can’t show us them. Apparently the camera he uses can’t be swiveled to show a screen. C’mon, Robert — if we’re investing the time (over a half-hour) to watch your video, make use of the medium.

He talks about a search for HDTV on Google versus Mahalo and pokes at the Google results, despite Google showing Wikipedia, How Stuff Works, CNET as well as a page from Amazon, just as Mahalo does.

I mean seriously, you might want to ding Google for deciding Amazon deserves to have a top listing for HDTV over all the other types of information that could be shown (especially if you’re one of the millions who use Google from outside the United States). But if you ding it for that, what’s up with Mahalo’s supposedly great human method of also deciding Amazon deserves a top spot.

Mahalo lists more than a top seven list, of course — you get a chunk of review sites, manufacturers, retailers and so on. But here’s the deal on Mahalo – it’s not really a search engine. The page it provides is good human crafted content, a good destination page like you might find at some of the other destination pages that Google lists. Mahalo — as Jason Calacanis himself will tell you — is a great place to start searching if your searches involve very popular queries. But if you want to hit those "search tail" terms that people always encounter? It’s not going to help.

My desktop computer crashed today just when I got back from a trip, and since it’s likely to be down for a day or two, I decided to start using a new Vista laptop I purchased until I can buy a Mac replacement (heh — well, maybe not). But ZoneAlarm for Vista doesn’t block http referrer information as it does on Windows XP. That led me to do a tail search — block referrer plugin for firefox — something that’s only going to happen a few times per month, relatively speaking. Try it on Google, then on Mahalo. Plenty of good solutions right at the top on Google. On Mahalo, I have to wade through four "related" links that aren’t relevant (Ryan Block, Sunscreen, Netscape and Jet), then I get Google results.

Robert also tells us that Mahalo rocks because you know, the first thing you do in the search process for HDTVs is want to know the manufacturers. Bull.

First of all, no one can predict how someone else will go through a search process, so that’s bull strike one. But if I want to play magic mindreader like Robert, I’ll say the first thing people want are some guides to HDTVs. What is an HDTV? Is 720p enough to make a TV HD quality? Does it have to have HDMI?

Saying you first go to a manufacturer site in the process is like saying that if you want to buy a new car, just go visit some car dealerships. Me — I go to Consumer Reports, figure out the cars I might want from a third party trusted resource, understand the jargon I might encounter, and then I go to the dealership. And as someone who bought an HDTV last year, I also remember going to the horrible manufacturer web sites where they often provided only sparse info about their own products and certainly didn’t compare them to competing products.

We then learn from Robert that Google can’t change to be like Mahalo because it has algorithms that are "stuck in sand, stuck in cement" and shifting will "prove impossible."

Insane. Seriously, like you want to scream stop talking. Robert’s a personally likable guy, but watching him make statements like this is like watching someone driving a car full speed toward a concrete wall while yelling "It’ll be OK — we’ll get through."

Google has constantly changed its ranking algorithm over the years and will continue doing so. If Robert knew any SEOs, they’d tell him this firsthand. But more to the point, Google can’t change the ranking algorithm to be more like Mahalo because Mahalo isn’t using an algorithm to rank web pages — it’s using human editors. Maybe Google someday might get an algorithm to mimic much of what Mahalo does, but that still wouldn’t be the same as using actual humans.

So it is impossible for Google to change! Maybe the algorithm, but Google could easily hire editors of their own, pay them more than Jason does and do what Mahalo is doing if that model takes off — which, so far, hasn’t happened.

Robert then jumps into the idea that Google also can’t integrate social networking into its algorithms, pointing to Google’s largely failed Orkut social networking site as an example. He completely overlooks the fact that Google is playing the human/social aspect on a different level — personalized search, where results are refined based on what you as an individual seem to like. That’s a major shift for Google, and it’s also one that I’ve found personally compelling. For more on the service, see:

• Google Ramps Up Personalized Search
• Just Behave: Google’s Marissa Mayer on Personalized Search
• Google Search History Expands, Becomes Web History

In particular, Google has been talking about how personalized search allows for creating personalized PageRank (and see here for a patent look), a way where rankings revolve around what you personally like. It’s not a hard leap to extend that into a "social network PageRank" model, where if you define a social network, the collective interests of that network could be used to model the rankings. Google’s not doing that now, but to suggest that the mechanism are somehow impossible from either a company attitude or technological model is simply being ignorant of Google.

Finally — halfway into part two of his video, 23 minutes of covering all that’s "wrong" with some existing players, Robert unveils how social might be blended into Facebook, giving you the impression this is simply a "please hire me" pitch to Facebook itself.

First step from Robert, use old-style on-the-page ranking. Yeah, there’s a waste of time.

Here’s a thought — why not just license an existing search engine period? I mean, how do you search the web when on the Facebook site itself? You don’t. The Facebook search box only searches within Facebook (and despite claims from Facebook itself that it is some type of fantastic people search engine, I’ve found the search less than compelling).

Facebook is partnered with Microsoft, so it’s somewhat amazing (if not telling) that there’s no ability to search using Microsoft’s Live Search. Hit MySpace, in contrast, and you’ll see how the Google partnership has Google web search over there.

Facebook doesn’t need to build on-the-page ranking from scratch, not to mention the nightmare situation of trying to crawl billions of pages.

Next, Robert gets into the social network and trust aspect. Sure — an exactly like what Eurekster and Yahoo already promised. There’s nothing new here. Well, there is. As Facebook has grown, we’ve also had frustration grow – including the famed Facebook Bankruptcy that Jason Calacanis declared last month.

People have friends on Facebook who aren’t friends at all. It’s just easier to accept them. Robert, at the time of this writing, has 4,875 "friends" in the system. Really — he knows all of these people? And wants all of them influencing his searches?

Ah — but see, Facebook knows how to "lock out" the SEOs, Robert tells us, so he’s not overwhelmed by noise. Sure — but on the flipside, Robert is one of the top FBOs out there, Facebook Optimizers, to the degree people have been complaining about how his activities dominate the news updates that Facebook sends out to those with him as a friend.

There will be more FBOs, no doubt about that. Any system that has lots of traffic will attract people who will study ways to tap into that traffic. That’s good and bad. It’s good in that since it’s going to happen, you want people to learn appropriate ways to do this. It’s bad in that there will no doubt be spamming that comes along with it.

For such hype about his video, I was pretty much left with a "is that it" response? Facebook will get pages, then look at a social network and hopefully get those people to proactively rank pages when they search. Despite the fact that the Yahoo My Web experience tells me people don’t want to build search results — they just want to search.

Social network data applied to search does have promise. But to assume that social networks can’t be spammed and lack noise is foolish. To assume that people want to participate in actively shaping results is also mistaken, in my view. To also assume that major players like Google or Yahoo can’t tap into things that make Mahalo, Techmeme or Facebook good is shortsighted. Yahoo Answers is akin to Mahalo. Google News is Techmeme across multiple subjects.

By the way, Robert, if you’re tired of the SEO "noise" you think screws up results, then do this. In a search for scoble, for well over a year now, you’ve crowded out variety in the results by not redirecting your scoble.weblogs.com address to your new home at scobleizer.com. That means you get both results 1 & 2 for your new place as well as results 3 & 4. You also have no content at scobelizer.wordpress.com, plus another version of your old place at radio.weblogs.com.

You have contacts with the Weblogs folks — getting a redirect should be easy for you. You can kill or block that Wordpress site that you no longer use. I assume you maintain these other sites simply so that when people search for you by name, you crowd out anyone else from ranking well, perhaps people who might disagree with you on topics. That’s an aspect of SEO — it’s a tactic used as part of public relations in SEO. It introduces the same "noise" into the results that you cheered about not being present in Mahalo. So clean it up or cut it out with the SEO slams. You’re doing SEO yourself.

What about that New York Times article I mentioned? When Bad News Follows You is the article, another amazing "SEO sucks" story. The New York Times has opened its archives for crawling, which apparently is causing people to come forward at the not so astounding rate of one person per day complaining about articles casting them in a bad light. Blame SEO:

Technically complex, search engine optimization pushes Times content to or near the top of search results, regardless of its importance or accuracy.

Wow, seriously — did I just read that in the New York Times? SEO just shoves whatever crud it wants to the top of Google. Hey, think there are some SEOs out there that would like to rank for "new york times." I guess they just need to SEO up some pages and they get there. Not.

Geez. The rest of the article does some hand-wringing on what to do to make the "right" articles appear at the top of the results (why not just sprinkle some SEO fairy dust on them?).

Insane. If an article is factually incorrect, then correct it. If the article is about someone with a negative connotation, then a later article comes out updating the story, link prominently from the top of the negative article to the latest version of a story. It’s called online journalism in the 2000s.

Postscript: I purposely haven’t read any of the other commentary on Robert’s post until I could brain dump my own thoughts. Since then, I read Rand Fishkin’s I Used to Respect Robert Scoble’s Opinion post, and he does a great job of poking back at the SEO attack as well as debunking Robert’s ideas on that Maholo-Google search shoot-out I mentioned.

Google and search from Dave Winer points out that spam is not the problem that both Jason and Robert like to paint it as. There’s spam, but there are lots and lots of great results, too.

Why Google Should be Scared of Facebook from Dare Obasanjo highlights that Facebook’s wall around its content is a threat to Google, but that’s a wall I think will get ripped down sooner rather than later, if only when Facebook decides it needs to show more ads on those pages and so needs more traffic.

Techmeme has much more commentary.

Mahalo Follow is a new toolbar that allows you to view Mahalo’s human-powered search results next to the results from the major search engine of your choice or have them appear in response to pages you view on the web.

Mahalo Launches With Human-Crafted Search Results from me in May explains more about the Mahalo service — Jason Calacanis’s challenge to Google and the other major search engines that human editors can craft better results than algorithms. But nearly two months after its launch, fair to say that the major search engines aren’t shaking in their boots, as Mahalo has yet to gain any serious traction.

Enter Mahalo Follow as a solution. Understanding that searchers just aren’t going to give up their favorite search engines, Mahalo Follow is designed to complement regular results.

"It’s extremely hard to get off of Google. I’m having a hard time, and I run a search engine," Calacanis told me, when talking about the new toolbar. "It might be a fool’s errand to get people off. If you can’t get them off, maybe you can get them adjacent."

Do a search, and the toolbar puts Mahalo results next to the search engine you used. For example, here’s a search for "simpsons movie" done with Google:

I did that search via the Groowe toolbar in my browser (a great toolbar that I highly recommend). I could have done it using the Google Toolbar, Firefox’s native toolbar or from the home page of Google or another search engine. The result would be the same — Mahalo checks to see if it has a match for what you searched for and, if so, opens up a sidebar window showing its results.

It’s pretty smart only to open if it has something solid. A search for "scouting" or "climbing walls," for example, didn’t make the window open.

Calacanis hopes that when people see his Mahalo results side-by-side with regular search results, they’ll begin to consider Mahalo more for making searches initially.

The idea of sidebar comparison search like this has been done before. Lycos SideSearch was launched in July 2003 to show Lycos results next to those of other search engines. That product never took off. Search for it today, and you mainly find pages from people who considered it spyware and were trying to get it off their computers (I can’t even find that the product has an install page any longer, though to serve its results remains here).

Perhaps Mahalo’s results will be so great that word-of-mouth gets everyone to install Mahalo Follow, so that it doesn’t follow the route of Lycos SideSearch. Then again, Jason’s got another idea — how about bribes? There’s a contest that will reward people who get others to download the toolbar and conduct searches.

One twist Mahalo Follow has over Lycos SideSearch is that it will also show you information as you surf, as opposed to just reacting to searches. For example, below I’m reading an article about Elton John deciding the internet should be shut down to inspire more creativity:

Off to the side, Mahalo helps me understand who this insane person is. I get links to his official site, his Wikipedia entry, a video clip and more.

Again, though, Mahalo is following in footsteps already tried. The Alexa Toolbar used to have a sidebar to show related information about pages you viewed. This information is still offered, though now via the toolbar’s drop-down menus.

Of course, Mahalo maintains the twist over both Lycos and Alexa in that the information it provides is human-powered, rather than machine generated. Still, to come up with those pages, it has to automatically scan them, find the words it thinks are most important and use that to generate sidebar results. That’s far from perfect.

For the Vans web site, Mahalo Follow tells you at the bottom of the sidebar that it found these words:

post, comment, before, login, must, xgames, silver, bmx, gold, team, grab, skatevanscom, week, clip, password, permalink, sign, check, forgot, media

As a result, it suggests topics such as American’s Most Wanted, Terrorism and Violent Crime Rate:

Dude, skateboarding is not a crime! Nor a terrorist activity :)