Cuil Fast Test - Relevancy Isn’t A Google Killer

Now that Cuil is live, I wanted to do a few fast queries to get a sense of how it seems to stack up against Google. With the huge caveat that nine queries are far from letting anyone conclude anything, I still didn’t come away with a sense that Cuil has Google-beating relevancy. Instead, it has some flaws though, admittedly, is better than many start-up search engines appear out of the box. Queries I tried:

miserable failure: Cuil has suggested it wouldn’t be vulnerable to the Googlebomb that made President George W. Bush rank at the top of Google for years for this phrase. Not so. Bush shows up at number two (reading right to left).

cars: Cuil said they’d feature a variety of subjects for a query. So for [cars], I’m wanting to see pages for both automobiles and the movie "Cars," as I get with Google. This doesn’t happen. Indeed, the movie doesn’t even appear as a related category to explore.

jaguar: Another good query for testing if a variety of subjects come up, as jaguar can refer to the game console, the car or the animal (among other things). Cuil gives me all three, while Google misses the gaming system. But Cuil puts me off with this description for the "Save The Jaguars" page:

[about jaguars] [conservation] [adopt a jaguar] [news] [resources] [donate]

That’s easily fixed, but descriptions are important for generated relevancy among searchers. I’m also not certain why schrodinger.com comes up in the top results.

viagra: Seeing URLs for utahpheasantsociety.org (which redirects to a Viagra selling site) or a page called "EuroDNS & Domain Name Parking Page" for Cuil’s results are off-putting, but they do have the official site as well as the FDA. Of course, Google has both, and with the FDA it points right at the FDA’s page on Viagra.

patents: Getting patents.cos.com first, then being told that patent search is no longer available there and to visit the US Patent Office, is disappointing. Getting the US Patent office listed fifth is also disappointing. Getting the Wikipedia Page on the Google Patents service, rather than patents in general, yet another disappointment. Google lists the US Patent office first and the general Wikipedia page on patents third.

new york: Well, getting only three results tells me something’s just not right with Cuil at launch!

danny sullivan: Ego search time! Ego searches are a terrible way for anyone but an individual to usually measure relevancy, but they’re often done and search engines are assessed individually on them. For me, it’s all me! The race car driver isn’t to be found! He gets one listing on Google.) Cuil lists the old place I used to write at (Search Engine Watch) first, rather than Search Engine Land (as Google does — the old place comes up third). My personal blog at daggle.com isn’t listed — Google has it. Both have my Wikipedia page. Google has my Twitter page and my main page from my consulting web site.

annie williams watercolour: One of my favorite artists; in the past I’ve found virtually little about her on the web. This search gives me 14 results from Cuil, though only 7 are listed. Her Wikipedia page doesn’t come up, probably because it spells "water colour" as two words. Google, supposedly less comprehensive, gives me 777,000 matches! OK, skip the numbers — something’s probably not right with Cuil’s counting. Still, Google gets me her Wikipedia page plus a BBC Radio 4 interview — something Cuil doesn’t have at all.

17 palms oasis: A cool place in California’s Anza Borrego dessert where people leave notes to each other in a diary box under one of the palms. Cuil’s first page of results all look very on-target and relevant. Google’s are also good, though the last about Kauai isn’t on the main subject I had in mind.

Let me end with a caveat again. To measure relevancy, you want to run a battery of tests on a variety of different subjects. Lots and lots of them, and have a number of people share their own subjective experiences. My article from 2002, "In Search Of The Relevancy Figure," goes into depth about this. Despite my call then and my 2005 article, "Screw Size! I Dare Google & Yahoo To Report On Relevancy," we still lack this type of metric or testing body. As a result, a different set results could show Cuil (or another search engine) as more relevant than Google – or not. But the lack of such testing means that no one (not even Google) can authoritatively claim that title.