Cuil Fast Test – Relevancy Isn’t A Google Killer

Now that Cuil is
live, I wanted to do a few fast queries to get a sense of how it seems
to stack up against Google. With the huge caveat that nine queries are far
from letting anyone conclude anything, I still didn’t come away with a sense
that Cuil has Google-beating relevancy. Instead, it has some flaws though, admittedly, is
better than many start-up search engines appear out of the box. Queries I
tried:

miserable failure: Cuil has suggested it wouldn’t be vulnerable to
the Googlebomb that
made President George W. Bush rank at the top of Google for years for
this phrase. Not so. Bush
shows up at
number two (reading right to left).

cars: Cuil said they’d feature a variety of subjects for a query.
So for [cars], I’m wanting to see pages for both automobiles and the movie
"Cars," as I get with
Google. This doesn’t happen.
Indeed, the movie doesn’t even appear as a related category to explore.

jaguar: Another good query for testing if a variety of subjects
come up, as jaguar can refer to the game console, the car or the animal
(among other things).
Cuil gives me all three,

while Google misses the gaming system. But Cuil puts me off with this
description for the "Save The Jaguars" page:

[about jaguars] [conservation] [adopt a jaguar] [news] [resources]
[donate]

That’s easily fixed, but descriptions are important for generated
relevancy among searchers. I’m also not certain why schrodinger.com comes up
in the top results.

viagra: Seeing URLs for utahpheasantsociety.org (which redirects
to a Viagra selling site) or a page called "EuroDNS & Domain Name Parking
Page" for Cuil’s
results are off-putting, but they do have the official site as well as
the FDA. Of course, Google has both, and with the FDA it points right at the
FDA’s page on Viagra.

patents:
Getting patents.cos.com first, then being told that patent search is no
longer available there and to visit the US Patent Office, is disappointing.
Getting the US Patent office listed fifth is also disappointing. Getting the
Wikipedia Page on the Google Patents service, rather than patents in
general, yet another disappointment. Google lists the US Patent office first
and the general Wikipedia page on patents third.

new york: Well,
getting only three results tells me something’s just not right with Cuil
at launch!

danny sullivan: Ego search time! Ego searches are a terrible way
for anyone but an individual to usually measure relevancy, but they’re often
done and search engines are assessed individually on them. For me, it’s all me!
The race car driver isn’t to be found! He gets one listing on Google.) Cuil
lists the
old place I used to write at (Search Engine Watch) first, rather than Search
Engine Land (as
Google does — the old place comes up third). My personal blog at
daggle.com isn’t listed — Google has it. Both have my Wikipedia page.
Google has my Twitter page and my main page from my consulting web site.

annie williams watercolour: One of my favorite artists; in the
past I’ve found virtually little about her on the web. This search

gives me 14 results from Cuil, though only 7 are listed. Her Wikipedia
page doesn’t come up, probably because it spells "water colour" as two words.
Google, supposedly less comprehensive, gives me 777,000 matches! OK, skip
the numbers — something’s probably not right with Cuil’s counting. Still,
Google gets me her Wikipedia page plus a BBC Radio 4 interview — something
Cuil
doesn’t have at all.

17 palms oasis: A cool place in California’s Anza Borrego dessert
where people leave notes to each other in a diary box under one of the
palms. Cuil’s first page of results all look very on-target and relevant.

Google’s are also good, though the last about Kauai isn’t on the main
subject I had in mind.

Let me end with a caveat again. To measure relevancy, you want to run a
battery of tests on a variety of different subjects. Lots and lots of them,
and have a number of people share their own subjective experiences. My
article
from 2002, "In Search Of The Relevancy Figure," goes into depth about this.
Despite my call then and my 2005
article,
"Screw Size! I Dare Google & Yahoo To Report On Relevancy," we still lack
this type of metric or testing body. As a result, a different set results
could show Cuil (or another search engine) as more relevant than Google —
or not. But the lack of such testing means that no one (not even Google) can
authoritatively claim that title.