Ask.com To Focus On Q&A Search, End Web Crawling

Talk about getting back to your roots. Ask.com, which originally used humans to find answers to questions, is going further back to its origins by abandoning its web crawling technology.

The Need, Or Not, To Crawl

Search engines like Google and Bing try to have answers to anything someone might search for by “crawling” the web, making copies of all the pages they find and storing them in a searchable database, or index. Ask has done the same thing since around 2001, when it acquired the Teoma search engine.

Crawling helps ensure that you have a comprehensive index. However, crawling is an expensive job — as is maintaining ranking algorithms, the systems that sift through the billions of pages stored in an index to select the best ones in response to a search.

For Ask, it no longer makes sense to maintain that technology. Earlier this year, the company shifted from trying to compete directly with Google and Bing as an all-purpose search destination to instead being a place where people can do Q&A search (see Ask Comes Full Circle With “Q&A” Offering). Running its own comprehensive web search system wasn’t helping with that mission.

“In some ways, we built a hammer and were looking for nails. We realized to the extent that they were doing general web search, our flavor of web search and the other favors out there, we weren’t adding much using our own technology,” Ask.com president Doug Leeds told me.

Outsourcing For Search

Ask will continue to use its web crawling technology, but far more selectively. Rather than trying to find everything from across the web, the crawling will be much more focused around sites that provide answers to questions people search for at the service. Ask will also continue to maintain its own news search service, both through crawling and pulling in news feeds.

Of course, Ask doesn’t want to be in a position where if someone does a search, they come up empty if Ask’s own database of answers has nothing. So the company will outsource for the comprehensive web search matches that it used to gather itself.

Which company will provide those results? Leeds said he’s not allowed to say. Almost certainly, it’s Google. Ask already has a deal with Google through 2012 to carry some of Google’s ads. There have also been reports that Ask has tested using Google’s results in the past, though Ask has never confirmed these and at one point, flat out denied it.

The outsourcing also means layoffs. Leeds said Ask will consolidate its operations to Oakland, California. It has about 60 people focused on search technology in Edison, New Jersey. About 1/3 of these will be offered relocation, with the remainder let go. It also has about another 60 based in China, where everyone is being laid off.

Postscript: I asked Google and was told:

We cannot comment on our partnership with Ask.  Ask has been a long standing partner of Google and we look forward to continue working with them in the future.

I also asked Bing if they were powering Ask and was told that it had “nothing to share on this.”

That was odd. I see no reason for Bing to clarify that they are not powering Ask. Heck, I see no reason for Google not to confirm it, other than perhaps it doesn’t want to attract attention that overall, it has even more share of the US search market than might be known (only about 3% more, but still).

Switching Over

When does the technology switchover happen? Leeds said that Ask has been testing using its search partner’s results for some time, and there’s no actual “on/off” switch that will be flipped for its own technology. Instead, Ask will blend results together, as it makes sense initially. Over time, more of the partner’s results will show.

“I don’t think you’re going to see any particular difference in our results versus tomorrow or next week.”

Earlier this year, Ask also brought back its Teoma web site, which featured “pure” search results from Ask’s Teoma search technology. Leeds said the company hasn’t decided what to do with the site yet.

Moving On With Q&A

Ask marks the second major search engine this year to give up its own search technology. Yahoo did so earlier this year, outsourcing to Bing in August. I’m sad that we’ve lost yet another search “voice,” though the launch of Blekko last month provides some optimism, on that front.

I’ve been pretty dubious about Yahoo’s chances of success in maintaining search market share. Major players that have given up their own search technology typically haven’t stayed in the majors. Yahoo has argued that search is just a “chip” like that in a computer — and what sells computers isn’t the chip inside but the overall experience. We’ll see if that metaphor plays out. I remain doubtful.

I was also pretty dubious about Ask’s future way back in 2008, when it had a leadership change and wasn’t clear about what its next move would be. As a major search player, I think that remains true. Ask isn’t going to build itself into a Google-killer, much less a Bing-killer. Barry Diller, CEO of IAC which owns Ask, recently admitted as much himself.

Over at Bloomberg, Brad Stone has a more detailed look today’s move by Ask from a business perspective, IAC’s Diller Surrenders to Google Juggernaut, Ends Ask.com Search Effort.

As a Q&A service, Ask may have some potential. Certainly it plays to a niche, and one that it still is known for despite having, until recently, really been out of the Q&A world for years. Since the reemphasis on question answering, Leeds said the service has gone from 30% to 60% of questions being answered.

Leeds also provided some stats from internal brand surveys that have been done. In one, Ask ranked well ahead of Google as being thought of as a place to get questions answered (46% to 27%):

In another, asking questions was a top reason for the most recent search someone did at Ask (20%) versus looking for information about a planned purchase at Bing (12%), Google (11%) or Yahoo (9%):

“No one is taking the Q&A communities and blending with Q&A search,” Leeds said. “My charge to people at Ask is ‘Great, let’s build it’.”

Of course, one of the big challenges I see in the Q&A space is the incredibly poor answers that some of them provide. I’ll be doing a future piece on this, but it’s common to get incorrect answers or no answers at all. It’s somewhat alarming, actually, how much “answer farm” content is polluting the results I find at places like Google and Bing.

Leeds also talked about former Ask CEO Jim Lanzone being right about focusing on retention and loyalty of Ask’s existing base — and it is a base, he says, that wants Q&A.

“If we increase our frequency and retention of existing users, that’s a huge market opportunity for us,” Leeds said. “The product they want, and they’re telling us they want, is to get more answers to their questions.”