Researcher Jim Jansen On The Truths & Myths Of The Search Buying Funnel

Jim Jansen is one of the few academics I know that is fascinated with Internet search. He has spent a good part of the last decade looking at patterns in search query and website logs, dissecting them and continually looking for significant trends. Jim and I crossed paths a number of years ago and have kept in touch ever since. When Chris Sherman asked someone to talk to Jim about his latest research, I was quick to volunteer. As I said in my last column, a chat with Jim is always fascinating.

First, a quick word about Jim’s background. Dr. Jansen is an associate professor in the College of Information Sciences and Technology at Pennsylvania State University. He has more than 150 publications to his credit. In fact, in the time I’ve know Jim, he has turned out papers at an amazing rate. He’s also co-author of the book, Web Search: Public Searching of the Web and co-editor of Handbook of Weblog Analysis. Jim moved into his academic career from the military, where he taught at West Point.

Jim’s research project

Jim has spent the last few years working on massive data sets that have been made available by large, high traffic sites. He has taken a statistical approach to dissecting these data trails and gaining insight into behavior through that analysis. You could say that Jansen is investigating what John Battelle has called the database of intentions.

The database of intentions was a concept Battelle introduced in his book The Search, How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture. In his words, the database of intentions is:

The aggregate results of every search ever entered, every result list ever tendered, and every path taken as a result. It lives in many places, but three or four places in particular hold a massive amount of this data (ie MSN, Google, and Yahoo). This information represents, in aggregate form, a place holder for the intentions of humankind – a massive database of desires, needs, wants, and likes that can be discovered, subpoenaed, archived, tracked, and exploited to all sorts of ends. Such a beast has never before existed in the history of culture, but is almost guaranteed to grow exponentially from this day forward. This artifact can tell us extraordinary things about who we are and what we want as a culture. And it has the potential to be abused in equally extraordinary fashion.

As Battelle admitted, this was a BIG IDEA. Jansen has taken a methodical approach to slicing and dicing his own subsets of the database of intentions. I’ll be talking to Jim about one of these slices in today’s column, but first, Jim will explain a little more about his current project:

Jansen: I have several research projects going on. One that I really find interesting is analyzing a five calendar year search engine marketing campaign from a major online retailer and brick-and-mortar retailer. It’s about 7 million interactions over that time, multi-million dollar accounts and sale. A fascinating temporal analysis of a search engine marketing effort. I’ve been looking at that at several different levels—the buying funnel being one, aspects of branding being another, and then the aspect of some type of personalization, specifically along gender issues.

The buying funnel and search engine strategies

The slice of Jim’s research I want to focus on in this column is the idea of a search “buying funnel.”

Marketers have long cherished the buying funnel model. The foundations of this model go back to the AIDA model—Attention/Interest/Desire/Action—introduced by Elias St. Elmo Lewis in 1898. The roots of this model run deep. The labels of the stages of the funnel vary somewhat, but generally they align with Need—Awareness—Consideration—Purchase. In my own recent research in the B2B marketplace, I have expressed some doubt about the applicability of the funnel as a workable model. I don’t dispute the stages, but I do question the idea of a linear “funnel,” with prospects moving obediently from one stage to the next. Based on my observations, many purchases are just not that simple.

But if we accept that some sort of buying funnel is in place, and we know (because we do know) that search is used to qualify and research buying decisions, than it makes sense that there should be a corresponding search funnel. Jansen went in with this basic hypothesis:

Jansen: One goal was to verify whether the buying funnel was really a workable model for online e-commerce searching or was it just a paradigm for advertisers to get their handle around this chaos. And the other goal was to discover if it’s an effective model, what can it tell us in terms of how advertisers should respond?

Jansen’s not the first to explore the territory of a search funnel. A comScore study of search behavior in consumer electronics in 2004 questioned the existence of a search funnel:

“The results of the study challenge a widely held belief that most consumers begin the product search process by using a generic search term (e.g. “plasma TV”) and then later refine their search activity to product-specific terms (e.g. “Sony Plasma KE-42M1″). Operating under this assumption, many retailers and manufacturers believed that investing only in product-specific terms allowed them to reach the majority of in-market consumers closer to their purchase decision. In reality, by taking this approach, marketers are missing the vast majority of their addressable market, since most consumers never use these types of terms.”

So, what did Jansen find in his data set?

Jansen: In terms of the first question, we had some mixed results. One, at the individual query level you can classify individual queries into different levels of this buying funnel model. There are unique characteristics that correspond very nicely to each of those levels. So in that respect, I think the model is valid.

Where it may not be valid is specifying this process that online consumers go through. We found that, no, it didn’t happen like that. There was a lot of drop-out and they would do a very broad query and then there may or may not be more specific queries after.

So we looked at the academic literature—what theoretically could deal with that or explain that?—and the idea of sufficing seemed to fit. If it is a low cost, they won’t spend a lot of time searching… they will just purchase it and buy it.

In terms of classifying queries in terms of what advertisers’ payoff is, I think the most interesting finding was that the purchase queries, the last stage of the buying funnel, were the most expensive and had no higher payoff than the awareness or the very broad, relatively cheaper queries. From talking to practitioners, that is a phenomena that they have noted also… which is why a lot of people bid still on very broad terms, to snatch these potential customers at an early stage.

Jansen’s findings seemed to support the earlier comScore findings and showed that search activity, just like consumer activity, doesn’t go in predictable or logical straight lines. I dove deeper on this particular area with Jim:

Hotchkiss: We similarly have found that you can’t assume a search funnel is happening because people use search at different stages and they’ll come in and then they’ll drop out of the process, and they may come in later or they may not, they may pursue other channels. But the other thing we found is sometimes there’s a remarkable consistency in the query used all the way through the process and we that quite often can be a navigational behavior. It can be people who say, “Okay, the last time I did this, I searched on Google for so-and-so and I remember the site I found was the third or fourth listing down,” and they use the same route to navigate the online space over and over again. So if you’re looking at it from a pure query level, it’s a bit of a head-scratcher because you’re saying, “Why did they use the same query over and over?” Again, it’s one of those nuances of online behavior. Did that seem to be one of the possible factors of some of the anomalies in the data?

Jansen: Well, that trend or something similar to it has been appearing in a lot of different domains and researchers are kind of attributing it to “When I do a query, I expect a certain result.” So with a query that may be very informational, what we’re finding is that searchers expect a Wikipedia entry… a very navigational intent behind that very informational query. And I think the phenomena you’re describing is very similar. We have a transactional-type query and users are expecting a certain web page, a navigational aspect: “Okay, I have an anchor point here that I’m going to go to.” I just looked at a query log from a major search engine and an unbelievable amount of queries were navigational in nature.

The other area I found fascinating was this concept of “sufficing,” or, as Herbert Simon labelled it, “satisficing.” Jim and I used that as the jumping off point for a rather interesting discussion about satisficing and how it might play out on a search engine. I’ll just let the conversation play out here, as I think it’s self-explanatory.

Hotchkiss: You know, the idea of satisficing, of taking a heuristic shortcut with their level of research, is also interesting. It seems that if the risk is fairly low, the online paths are shorter. Is that what you were finding?

Jansen: Yes… the principle of least effort is how it’s also presented. We see it in web searching itself in how people interact with search engines and how they interact with sites on the web. They may not get an optimal solution, but if it’s something that’s reasonable and it’s good enough, they’ll go for it. And that seems to be occurring in the e-commerce area also. “I want to buy something relatively cheap. Okay, this particular vendor may not have the best price, but guess what? It’s close to what I’m thinking it should be. Just go and get it done, get it over with, buy it.”

Hotchkiss: And I would suspect that that would also be true in product categories where you mentally have a good idea of what an acceptable price range would be, right?

Jansen: Yes.

Hotchkiss: So if it’s a question of making a trade-off for $2 but saving yourself a half hour of time, as long as you’re aware of what those price ranges would be, you’re more apt to take that shortcut, right?

Jansen: Yes. It does assume some knowledge and risk mitigation—if it’s a small purchase. That varies a little bit for each of us, but you’re willing to cut your costs of searching and trying to find the best deal just to get it done.

Hotchkiss: Part of this too would be your level of personal engagement with the product category you’re shopping in. For instance, I’ll spend way too much time researching a purchase of a new gadget or something that I’m interested in just because I have that level of engagement. But if it’s basically a purchase that’s on my to-do list, if it’s just one task I have to get done and then move on to the next thing, I suspect that that’s where that satisficing behavior would be more common.

Jansen: Now you bring up a really good point. If it becomes entertainment— like a gadget that you enjoy researching—it’s no longer work, it’s no longer something you get done. The process of doing it makes it enjoyable so you don’t mind spending a lot of time. In those kind of cases, the goal really is not the purchase, the goal is the looking.

Hotchkiss: And we found that alters the behavior on the search page as well. If it’s a task-type purchase where I just have to go and get there, you see that satisficing play out on the search page too. Typically when we look at engagement with the search page, you see people scan the top four, three or four listings. It’s that satisficing type of intent where you say, “I just want to buy this thing.” What you’ll see is that people scan those first three or four and pick what they feel is, like you say, the path of least effort. They go down and say, “Okay… it’s a book. Amazon’s there. I know Amazon’s price. I’m just going to click through and order this,” but if it’s entertainment, then suddenly they start treating the search page more like a catalog where they’re paying more attention to the brands and they’re just… they’re using that as a navigational hub to branch off to three or four different sites. And again, it can really impact the nature of engagement with the website… or with the search page.

Jansen: Absolutely, and I really like your analogy of a catalog. You know, there are some people that love just looking at a catalog—flipping through it, looking at the dresses and shirts or gadgets or sporting gear or whatever. And that’s a much different engagement than flipping through the classified ads trying to find some practical thing you need. The whole level of engagement is at totally opposite ends of the spectrum, really.

Hotchkiss: As an extreme example of that, we did some eye-tracking with Chinese search engines and what we found was, with Baidu in particular, people using it to look for MP3 files to download. So when we first saw the heat maps—and of course it was all in Chinese, so I couldn’t understand what the content on the page was without having it translated—but I saw these heat maps going way deeper and much longer than we ever saw in typical North American behavior. We saw a level of engagement unlike anything we had ever seen before. And I said, “Well, what’s going on here?” and that was exactly it. It was a free task… they were looking for MP3 files to download and they were treating the search page like a catalog of MP3 files. So they were reading everything on the page and I think that’s just one extreme example of this catalog browsing behavior that we were talking about.

So let’s go to one of the other findings on the buying funnel which was that quite often the more general, broader categories from an ROI perspective can perform just as well as what traditional wisdom tells us is your higher return terms which are closer to the end of the funnel—they’re more specific, they’re longer, they’re more transactionally oriented. What’s behind that?

Jansen: Well, in a lot of these questions there’s no simple answer… there’s plenty exceptions to the rule of what you have just described there. There are some very broad terms that are very cheap, others that are very expensive. On the purchase side, there are some key phrases that are very cheap because they’re so focused and others are expensive. But in this particular analysis—and again, this was 7 million transactions over 33 months, from mid-2005 to mid-2008—the awareness terms were cheaper than the purchase terms and they generated just as much revenue.

I think a lot of it is that perhaps the items this particular retailer were selling fell into that sufficing behavior where they were gifts, fairly low-cost items. There was just no need to progress all the way to that particular purchase phase.

Really… it was very unexpected. I really expected those purchase terms to actually be cheaper because they were more narrowly focused and to generate more revenue, but overall… it didn’t turn out that way.

Hotchkiss: That brings up an interesting point that we’ve seen with client behavior, especially given the current economic condition. What we found is a lot of clients are tending to optimize down the funnel—they are looking at their keyword lists they’re bidding on and move further and further down to more and more specific phrases, because the theory is—and generally they do have analytics to back this up—that there’s greater ROI on that because these are usually people that are searching for a specific model or something which is a pretty good indicator that they’re close to purchase. But I think one of the by-products of that is that as people optimize their campaigns, those long tail phrases are getting more and more expensive because there’s more and more competition around them, and as people move some of their keyword baskets away from those awareness terms, maybe the prices on that, it all being based on an auction model, are starting to drop. Do you think that could be one of the factors happening here?

Jansen: That very well could be. The whole online auction is designed around [the idea that] as competition increases, cost-per-clicks will increase also. It also may be that those particular customers don’t mind clicking on a few links to do some comparison shopping and may end up going somewhere else… they may have a higher intent to purchase, but the competition among where they’re going to buy is more intense.

As always, I found the conversation with Jim enjoyable and enlightening. We continued to talk more about branding and search and the implications of personalizing to gender, both areas that Jim explored in this latest study. I’ll cover that territory in my next Just Behave column.