How To Overhype Your Search Engine
After covering search for 13 years, I’m more than a little jaded. I’ve seen any number of search start-ups promise to revolutionize how we search. None of them have in the huge way they’ve promised, other than Google — and it’s a special case. Over the past few weeks, I’ve been watching Wolfram Alpha walk the same hype path so many have followed before. So I thought I’d offer a little advice, for what it’s worth. Here’s how to set your search service up for a disappointing fall.
1) Fail to brief those in the know: First I heard of Wolfram Alpha was on Techmeme back in March. The company made no outreach to me. Big headed? Sour grapes? Hey, I haven’t seen John Battelle mentioning having talked to them, either. Nor have I seen anyone who regularly writes about search having said they know about the service. Last week, I even put in a press request for a briefing. Haven’t heard back. My takeaway when those who regularly cover search aren’t in the know? The company isn’t that clued in about search (given they haven’t done the basic research about who they should talk to) or they are clued in and are purposely avoiding the regulars who won’t be dazzled by controlled demos.
2) Use controlled demonstrations: The biggest red flag of a new service is when they trot out set examples of searches they say show how much better they are over the competition. This was a Powerset hallmark, and it’s been used by many others over the years. It’s the equivalent of running a poll where you’ve slanted all the questions to get the answers you want. Controlled demos mean that a service can pick-and-choose questions where it knows it excels. In turn, that can get some journalists believing that the new service really is better than existing ones. Then when it goes public, and has to face millions of queries that aren’t hand selected, it falls way short of expectations.
3) Claim you’re the “next Google” or fail to set expectations: Some companies are stupid enough to honestly believe they’ll unseat Google. Microsoft, spending hundreds of millions, hasn’t been able to knock people off the Google habit. A start-up just isn’t going to do the same. Google did it because of a unique moment. Search was a loss-leader, so the major search engines stopped focusing on quality. Now that search is a billion dollar industry, the focus is sharp. Some companies don’t explicitly say they’re going to beat Google but they allow others to do that on their behalf. Wolfram Alpha promises a “new paradigm.” Another review, following what I’m pretty sure was a controlled demo, tells me “most of us” will be using Wolfram Alpha by default. The story that broke the news about Wolfram Alpha specifically avoids calling it a Google killer but still suggests it will be as important as Google. When Wolfram Alpha does debut, it has huge expectations to meet. And chances are, it’ll fail to do so.
4) Trot out big names with no connection to search: A number of start-ups, over the years, have tried to prove they must be good by listing investors who made their names in other fields. I generally don’t care who has invested in you (unless you’ve got an investor with a track record of good search choices). The fact you’ve gotten investment doesn’t even reassure me, given my sad faith in the homework of most investors. Usually, when that’s your selling point, it’s a red flag you don’t have an actual product that’s compelling. (Postscript: Matt Cutts pointed out a classic example of this, when Bill Clinton was trotted out as the spokesperson for Accona
5) Offer me or anyone an “exclusive” to break your launch: I like being prebriefed. It helps in crafting a more detailed story for my readers. And sure, I like to break news as much as any journalist. But when a pitch overtly suggests that I can have the scoop over others, that tells me you’re not trusting enough in your product to let a variety of people review it. You’re hoping to control the story too much. It also tells me that you think I’m stupid enough to agree in advance to give your product heavy play even though it may not deserve it. For more, see my post, Embargoes, Prebriefings & Exclusives.
Search Engine Mouthfeel
These thoughts are also on my mind because earlier this week, I visited Blekko, another “stealth” search engine in development. I can’t go into detail about what I saw, but I was impressed. More important, the red flags aren’t going off. I got to search for whatever I wanted on Blekko. There was no controlled list of queries I had to do. There was no claim to be ready to topple Google. More than anything else, Blekko had good mouthfeel when I used it.
You know when you eat or drink something, and it feels just right in your mouth? That’s mouthfeel. I realized this week that search engines have mouthfeel, too. It’s an intangible thing where they just feel right or not. When I search on Google, it just feels right. On Yahoo, sort of. On Live Search, there’s no mouth feel. For some reason, the layout, the mixture, maybe the name — I don’t know. They just don’t feel right.
Microsoft’s Hype Challenge
Microsoft faces its own hype challenge, of course. A brand change is expected later this year, along with a new look and features. Microsoft is definitely taking on Google, but after years of failure — and some past pronouncements that it would soon beat Google — the company now has to diligently control the hype and continue stressing the long game it is playing. Otherwise, when it fails to gain huge market share six months after the relaunch — which almost certainly be the case (unless it buys Yahoo’s traffic) — there will be disappointment over how it failed to “beat” Google once again rather than positive attention on any smaller by significant gains it might make.
Waiting For Wolfram Alpha
As for Wolfram Alpha, it might be amazing when it comes out (Tim O’Reilly tells me I’ll be impressed, and I count his opinion pretty highly). But if so, it will go against the odds. So far, it just keeps tossing up the red flags on my list that lead to disappointment.
We’ll get a better idea on Tuesday, when a public sneak preview is being offered. If you watch the webcast, just keep asking yourself how controlled are the questions? Will the audience be able to suggest searches on their own? And how do those measure against the types of searches a typical search engine handles?
Knowing only things from afar, my guess is that Wolfram Alpha (which desperately needs a better name, another red flag) will be impressive in a niche that’s separate from what the existing search engines do. Not a replacement, and probably not a service that will attract a mass audience. But that’s not bad, if it does pan out that way. That’s still providing an important service, and potentially a profitable one. Not everyone has to be Google to be successful in search.
Postscript: There are some excellent comments below, along with some healthy discussion here on FriendFeed. The two main themes tend to be (1) am I just whining that I didn’t get briefed? and (2) Wolfram Alpha isn’t a search engine similar to Google. In fact, it’s a different creature altogether.
Sure, I suppose I’m whining. But the points above aren’t in order of importance, nor is it a case that Wolfram Alpha or any service that doesn’t do well has followed all of them. Instead, these are various types of red flags that go off for me, along with others (such as some relevancy signals you can see or interface issues, when you first use a service).
It’s not that I have to be briefed — it’s more that if I don’t see anyone who regularly covers search talking about a service, that’s usually not a good sign.
I don’t mean to sound big headed or insular in saying this. I’m just telling people that’s a regular sign I see of companies that aren’t that clued in, a sign that I tend to be sensitive to because of the unique position I’m in.
Being briefed doesn’t mean a company will succeed, of course. I have talked with plenty of companies ahead of time that think they’re going to be the next Google but fail. But usually, there are other red flags that have also gone off.
As for Wolfram not positioning itself as the next Google, certainly it suggested it was bringing in a new paradigm of searching. And reviews from those who have been briefed come away suggesting it could rival Google or be a service regularly used. So far, I don’t recall those services stressing this as something just for scientists. In fact, Rudy Rucker’s review gives queries such as “temperature in los gatos” as examples of what it solves — a query which, if you do it on Google, Yahoo or Microsoft, gets solved pretty well now. Rucker’s article also ends with Stephen Wolfram saying:
It will raise the level of scientific things that the average person can do. People will find that the world is more predictable than they might have expected. Just as running Google is like having a reference librarian to help you, running Wolfram|Alpha will be like having a house scientist to consult for you.
That’s not saying this is just for scientists. That’s saying Wolfram Alpha is specifically for “average people,” though qualified that they’ll want to use it for “scientific things.” Well, time and again, we’ve seen that “average people” don’t turn to specialized search engines to do specific tasks, not in mass numbers. Instead, they still go to the major players and try to make them perform. So yeah, it looks to me that Wolfram Alpha has some expectation problems they’re likely to face, and some that they’ve created directly themselves.
That doesn’t mean, as I said below, this won’t be a great service — or attract a key audience — or necessarily will have a problem with those expectations. So far, the business model seems non-existent. They may not need nor want mass numbers. But they will get compared to Google, and if they want to avoid looking like a failure if six months down the road they’ve failed to gain Google-like traffic (much less Microsoft-like traffic), then they need to do a better job of positioning what this service is and who it is for.
Too harsh when they haven’t even launched? No. Back in March, red flag #5 went off — an exclusive preview that positioned the service as something amazing that was coming. That review just didn’t happen — reviews like that don’t. Wolfram Alpha would have authorized the author to write, because they wanted a message to get out. That doesn’t mean the author necessarily delivered the message they wanted, but certainly there was little follow up I saw afterward to suggest they weren’t somehow going to be as important as Google.
But Monday will be an important new step. We’ll see more about what the service really is and how they hope to position themselves.
Postscript 2: See Impressive: The Wolfram Alpha “Fact Engine”, my review of the service which, as the headlines says, is indeed impressive. It also covers how it still is unlikely to be a Google-killer and the challenges to gain widespread use, though it has a better chance than many specialty services I’ve seen.