To Focus On Q&A Search, End Web Crawling

Talk about getting back to your roots., which originally used humans to find answers to questions, is going further back to its origins by abandoning its web crawling technology.

The Need, Or Not, To Crawl

Search engines like Google and Bing try to have answers to anything someone might search for by “crawling” the web, making copies of all the pages they find and storing them in a searchable database, or index. Ask has done the same thing since around 2001, when it acquired the Teoma search engine.

Crawling helps ensure that you have a comprehensive index. However, crawling is an expensive job — as is maintaining ranking algorithms, the systems that sift through the billions of pages stored in an index to select the best ones in response to a search.

For Ask, it no longer makes sense to maintain that technology. Earlier this year, the company shifted from trying to compete directly with Google and Bing as an all-purpose search destination to instead being a place where people can do Q&A search (see Ask Comes Full Circle With “Q&A” Offering). Running its own comprehensive web search system wasn’t helping with that mission.

“In some ways, we built a hammer and were looking for nails. We realized to the extent that they were doing general web search, our flavor of web search and the other favors out there, we weren’t adding much using our own technology,” president Doug Leeds told me.

Outsourcing For Search

Ask will continue to use its web crawling technology, but far more selectively. Rather than trying to find everything from across the web, the crawling will be much more focused around sites that provide answers to questions people search for at the service. Ask will also continue to maintain its own news search service, both through crawling and pulling in news feeds.

Of course, Ask doesn’t want to be in a position where if someone does a search, they come up empty if Ask’s own database of answers has nothing. So the company will outsource for the comprehensive web search matches that it used to gather itself.

Which company will provide those results? Leeds said he’s not allowed to say. Almost certainly, it’s Google. Ask already has a deal with Google through 2012 to carry some of Google’s ads. There have also been reports that Ask has tested using Google’s results in the past, though Ask has never confirmed these and at one point, flat out denied it.

The outsourcing also means layoffs. Leeds said Ask will consolidate its operations to Oakland, California. It has about 60 people focused on search technology in Edison, New Jersey. About 1/3 of these will be offered relocation, with the remainder let go. It also has about another 60 based in China, where everyone is being laid off.

Postscript: I asked Google and was told:

We cannot comment on our partnership with Ask.  Ask has been a long standing partner of Google and we look forward to continue working with them in the future.

I also asked Bing if they were powering Ask and was told that it had “nothing to share on this.”

That was odd. I see no reason for Bing to clarify that they are not powering Ask. Heck, I see no reason for Google not to confirm it, other than perhaps it doesn’t want to attract attention that overall, it has even more share of the US search market than might be known (only about 3% more, but still).

Switching Over

When does the technology switchover happen? Leeds said that Ask has been testing using its search partner’s results for some time, and there’s no actual “on/off” switch that will be flipped for its own technology. Instead, Ask will blend results together, as it makes sense initially. Over time, more of the partner’s results will show.

“I don’t think you’re going to see any particular difference in our results versus tomorrow or next week.”

Earlier this year, Ask also brought back its Teoma web site, which featured “pure” search results from Ask’s Teoma search technology. Leeds said the company hasn’t decided what to do with the site yet.

Moving On With Q&A

Ask marks the second major search engine this year to give up its own search technology. Yahoo did so earlier this year, outsourcing to Bing in August. I’m sad that we’ve lost yet another search “voice,” though the launch of Blekko last month provides some optimism, on that front.

I’ve been pretty dubious about Yahoo’s chances of success in maintaining search market share. Major players that have given up their own search technology typically haven’t stayed in the majors. Yahoo has argued that search is just a “chip” like that in a computer — and what sells computers isn’t the chip inside but the overall experience. We’ll see if that metaphor plays out. I remain doubtful.

I was also pretty dubious about Ask’s future way back in 2008, when it had a leadership change and wasn’t clear about what its next move would be. As a major search player, I think that remains true. Ask isn’t going to build itself into a Google-killer, much less a Bing-killer. Barry Diller, CEO of IAC which owns Ask, recently admitted as much himself.

Over at Bloomberg, Brad Stone has a more detailed look today’s move by Ask from a business perspective, IAC’s Diller Surrenders to Google Juggernaut, Ends Search Effort.

As a Q&A service, Ask may have some potential. Certainly it plays to a niche, and one that it still is known for despite having, until recently, really been out of the Q&A world for years. Since the reemphasis on question answering, Leeds said the service has gone from 30% to 60% of questions being answered.

Leeds also provided some stats from internal brand surveys that have been done. In one, Ask ranked well ahead of Google as being thought of as a place to get questions answered (46% to 27%):

In another, asking questions was a top reason for the most recent search someone did at Ask (20%) versus looking for information about a planned purchase at Bing (12%), Google (11%) or Yahoo (9%):

“No one is taking the Q&A communities and blending with Q&A search,” Leeds said. “My charge to people at Ask is ‘Great, let’s build it’.”

Of course, one of the big challenges I see in the Q&A space is the incredibly poor answers that some of them provide. I’ll be doing a future piece on this, but it’s common to get incorrect answers or no answers at all. It’s somewhat alarming, actually, how much “answer farm” content is polluting the results I find at places like Google and Bing.

Leeds also talked about former Ask CEO Jim Lanzone being right about focusing on retention and loyalty of Ask’s existing base — and it is a base, he says, that wants Q&A.

“If we increase our frequency and retention of existing users, that’s a huge market opportunity for us,” Leeds said. “The product they want, and they’re telling us they want, is to get more answers to their questions.”

Related Topics: Ask: Business Issues | Ask: Employees | Ask: General | Channel: Search Marketing | Top News


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Michael Martinez

    When I spoke with the DOJ about the Yahoo!-Bing deal they seemed to feel that Ask would take up the slack where Yahoo! lost search traffic. I told them I didn’t think Ask was up to the task but I really didn’t see this abrupt a departure from the search space coming.

    Blekko is at least 2 years out from achieving credibility as a search engine, unless they miraculously manage to buck historical trends. But they lack the marketing clout that Microsoft had when it entered the search business only a few years ago.

    Despite the fact we have nearly 100 search engines today, the barrier to entry into competitive search is extremely high. The market for all intents and purposes now comes down to Google and Bing, and that won’t change any time in the near future.

    We could say of Ask “That boy was our only hope” and then point out in Yoda fashion “No, there is another” but honestly there has been no hope of seeing a three-way split in search ever since Carl Icahn injected himself into Yahoo!’s internal business.

  • T Campbell

    The methodology that Ask originally touted was Q&A-based. It was made obsolete, like Yahoo’s directory-based offerings, by the continual improvements in query-based search.

    When it became about “the algorithm,” it was just another Google-of-the-future wannabe.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide