The Google Challengers: 2008 Edition

Rich Skrenta — who, aside from creating the first computer virus, is more
notable to search as a cofounder of the Open Directory Project and the Topix
news search engine — has announced he’s founded a search start-up. A stealth
one, as TechCrunch puts it. Don’t we already have several stealth search
start-ups? Yep. Here’s a guide to who’s who.

Blekko

What we know so far about Blekko isn’t
much, and TechCrunch has the most details in its

The Next Google Search Challenger: Blekko post from yesterday. Apparently
Rich founded the company in September 2006, along with five other former Topix
employees, after he left
Topix in June.

Rich told TechCrunch not to likely expect anything public until 2009. I agree
with Michael Arrington at TechCrunch that Rich has a track record that makes him
well worth watching. The Open Directory was
an initial success, though the model didn’t scale well. Some of that was within
the founders’ control but had
more
to do with AOL’s lack of backing. The company should be dragged into the
International Court Of Search Crimes and be forced to sell the ODP to someone
who will support it properly. Topix has
built a reputation and is still standing and succeeding — though I’d say it
still has far to go to seriously threaten Google or Yahoo.

Rich adds a bit more in his
Why Search? post
today:

Having just spent 5 years in the media space, I’ve come away with the idea
that editorial differentiation is possible. But the editorial voice of a
search engine is in the index…so it has to be algorithmic editorial
differentiation.

So far, it doesn’t sound like a social networking play like some of the others.
We’ll be watching, Rich. Also see discussion today
on Techmeme.

Powerset

Powerset is now a classic example of
why you WANT to be a stealth start-up and say little. That’s because when you
get too much early press — in part through your own doing — then fail to
deliver anything, the hype can swing back at you hard.

The company came to light back in October 2006

via VentureBeat, with the twist being that natural language search would be
the way forward. That caused me to write a
long rant
about the hype of natural language search in reaction. From the top of that:

This is a rant. It’s a rant from
over 10 years of watching people trot out natural language search as the
“killer” solution to the current state of search, something that’s happening
once again with
Powerset. That’s a search engine you can’t even use at the moment, but the
hype will no doubt continue. To counteract that, my thoughts on and some
history about natural language search.

Natural language search makes a compelling pitch for those who really
don’t know search or haven’t heard the natural language mantra before.
I’ve seen the pitch time and time again. You:

• Pick out an example that shows how “bad” search is on an existing
  search engine
• Demonstrate how natural language search would work better on your
  service
• Sit back and collect the press attention

I then went on to detail how natural language search had been hyped and tried
over the years. The short story is this: It doesn’t take much natural language
analysis to figure out what someone wants when they type in “britney spears
nude” or “hotmail.” In addition, by and large I don’t believe enough people will
change their basic search habits to enter long sentences when searching any time
soon.

Since that time, we’ve pretty much had nothing out of Powerset other than the
launch of Powerset
Labs in September 2007. That launch hasn’t produced any cool applications that
I’ve seen or heard about, nor much buzz. Instead, in November, we got a
management shake-up.

For a more formal chronicle of the company’s developments, check out
this area at
VentureBeat and these search
results at TechCrunch.

Finally, while I’m harsh above on Powerset, I actually had a long visit with
the company in the middle of last year and was deeply impressed with the effort
going on there. I’m still working on a long write-up to explain what’s
happening. But in a nutshell, Powerset is trying to literally comprehend or
understand each page on the web.

Today’s search engines don’t know what a page is about by reading words.
They’re more or less doing pattern matching — finding pages that contain words
similar to what you search for (or pages relevant to those words based on
linkage). Powerset literally is trying to read and understand what a page is
about the way a human reads a page and knows it is on various subjects.

I don’t see that as making it a better search engine that Google. Instead, I
think it may eventually give it the ability to create a unique “auto-Wikipedia”
style site, assembling knowledge pages on any subject automatically. I also
think that there will eventually be some search benefit in comprehension of
pages, but exactly how that will play out I suspect is part of being with an
existing search engine and a more traditional model. With the array of patents
Powerset has lined up, I suspect it will eventually get acquired by Google,
Yahoo, or Microsoft rather than rollout its own product. But we’ll see.

Hakia

Like Powerset, Hakia has played the natural
language search game. Unlike Powerset, it has a product anyone can use — live
since at least the middle of 2006.

Again, I’ve been working on a long write-up on the inner workings of Hakia
and have yet to finish it. It’s complicated, and I mainly want to cover what I
find to be the real use of their technology — the ability to create custom
“gallery” pages and understand those are related to particular searches.

It’s easier to show you what’s impressive. Search for
hillary clinton,
and you get a nice page showing news, her official site, biography pages, blogs
& fan sites, news & interviews, and more. It’s very Mahalo-like, except it
doesn’t require human editors like Mahalo and predates Mahalo by a year.

That categorization is something I know the major search engines could do, if
they wanted. So far, they don’t. And so far, despite Hakia talking about its
rising traffic, it has yet to make a
serious mark. Moreover, in October, it made a serious shift to allow social
interaction with its results. That’s a sign that the original plan that “natural
language will win all” has failed to do so; therefore, another twist is needed.

Social Networking
Through Search: Hakia Helps You Meet Others from Vanessa Fox here at Search
Engine Land covers the change, plus it gets into the natural language indexing
stuff I mentioned earlier that makes Hakia unique, plus has examples of gallery
pages.

Mahalo

Credit to Jason Calacanis. He said he wanted to take on Google, then wasted no
time getting Mahalo
rolled out. OK, he also says he’s not taking on Google — just focusing on the
top searches that he thinks would be better with human review. Sure, you aren’t
taking on Google, Jason.

To date, Jason reports that Mahalo’s traffic is growing and strong. But to
date, I’ve certainly see no webmasters taking about what a traffic driver Mahalo
is. It would be early to call it a raging success, but it’s a nice
alternative to have. Indeed, later this month I’ll finally finish my Search 4.0
piece that picks up from the conclusion of my
Search 3.0: The Blended
& Vertical Search Revolution article last November. I’ll show some examples
of how the human element at Mahalo can and has kicked some Google and
traditional search engine butt — though also how it isn’t the panacea some
expect.

Some of our
past
coverage of Mahalo:

• Mahalo Launches
  With Human-Crafted Search Results
• Mahalo Greenhouse:
  Get Paid For Writing Search Results
• Search Spam Fight
  – Mahalo: 1; Squidoo: 0
• Mahalo Follow:
  Toolbar Gives You Human-Powered Alternatives To Searching, Surfing
• The Promise &
  Reality Of Mixing The Social Graph With Search Engines
• Mahalo Adds The
  Social Graph To Search

Search Wikia / Wikia Search

Wikipedia founder (as he prefers to be called; Wikipedia itself calls him
cofounder) Jimmy Wales made waves a year ago when he said he’d take on “closed” Google
with humans and a transparent search engine. Called
Search Wikia (but, confusingly, it’s also called Wikia Search), Wales has grabbed attention from the press
over the past year. Slamming at Google as a
scary
closed thing gets you good mileage, especially when you helped establish
Wikipedia, a threat Google takes so seriously that it may launch its own
Wikipedia-style site,
Google Knol.

Now Wikia Search is at hand. A private “pre-alpha” test
started in late
December, an invite-only thing I still find odd for a service that’s supposedly
all about the “transparency.” But on Monday, the general public will finally get
a look at whatever Wales and his team have concocted. In the meantime, while
Wales still hasn’t posted any news since July 27 to the “news” section of Search
Wikia, press reports tell us so far:

• Only a tiny 50 to 100 million pages will be indexed at launch. The major
  search engines today have tens of billions of pages indexed. (AP)
• There will be a high degree of human editorial influence, though whether
  that’s over the algorithm or the search results on a per-query basis remains
  to be seen (CMP)
• An early
  screenshot suggested that Search Wikia might be evolving more into a
  Facebook-style service, perhaps with some ways for users to share results (Matthew
  Buckland &
  Wired)

Some of our
past
coverage of Search Wikia:

• Q&A With Jimmy
  Wales On Search Wikia
• Search Wikia Takes
  Steps To Crawl; Acquires Grub
• Search Wikia Gets
  Open Source Categorization Software
• Search Wikia
  Launches In 2007 With Private Beta

Cuill

Arguably the stealthiest of the stealth start-ups,
Cuill (pronounced “cool”) has an impressive
pedigree with its three founders: Tom Costello of IBM’s WebFountain project and
Anna Patterson and Russell Power of Google’s TeraGoogle project, its massive
search index. And last year, former AltaVista founder Louis Monier — who later
went to eBay as its first eBay Fellow, then to Google — jumped ship from Google
to join Cuill.

I talked with Cuill earlier this year to understand a bit more about what
they are doing, but the details are still being held very closely. The main
difference between Cuill and everyone else I’ve named above is that Cuill is
founded by people who understand and have dealt with firsthand the challenge of
indexing billions of documents.

Cuill recently

took on more funding. Louis is also going to be doing a
keynote at our
SMX West search marketing
conference, held in Santa Clara, California from Feb. 26-28. I’m thrilled to be having
him since there are only a handful of people who have worked for the “old”
Google (AltaVista), the current Google (when he was at the Big G), and a
potential future Google (Cuill).

And The Winner Is…

If you think the future of search is on smart automation, Cuill’s definitely
one to watch, and perhaps Blekko as well. If you think it’s the growth of
humans, Mahalo and Search Wikia are your better candidates. The reality is that
success will likely be a blend of the two. For the human services, a real open
source index would be a big help — see
Google: As Open As It
Wants To Be (i.e., When It’s Convenient) for more about this.

But the reality is that all of these services will have an incredibly tough
time to beat Google.

Google came along at a very special time, as I’ve long written. It had better
technology at a time when all the search engines had abandoned improving search,
since that was seen as a loss leader. The money was in portal features.

Today, search is a multi-billion dollar industry. If someone with a serious
search threat comes along, you buy them (such as with YouTube), or you start to
develop your own rival if it seems a real threat. Google’s not omnipotent — but
you’ve already got a space where it’s Google, Yahoo, Microsoft, and Ask all
seriously fighting it out (and the latter three, despite their funding and
experience, still struggle against Google as being synonymous as a trusted
search brand for most users).

To date, Google is the real exception of “a better mousetrap wins.” It’s far
more likely the companies above, if they do gain traction, will end up being
purchased for a large amount by one of the existing “search utility companies.”