In this column, I’ll follow up on my conversation with Dr. Jim Jansen from Penn State and his recent investigation into behavior patterns that lie within a large data set of visitor and search advertising campaign data from a high traffic ecommerce site. In part one, Jim and I explored whether a search funnel actually exists. Surprisingly, Jim found that more generic queries, considered by marketers to be “top of funnel” queries, may be the only search activity required. He found these terms tended to generate equivalent or higher ROI than longer, more transactional queries.

Today, I’d like to cover a couple of additional topics that came up in our conversation: personalization in terms of the “maleness” or “femaleness” of the query used, and how personalization may play out on both the desktop and on mobile devices.

Let’s start with the “sex” of queries. Jansen did an interesting segmentation of the queries in the dataset, using Microsoft’s demographic tool:

Jansen: We took queries from this particular search engine marketing campaign and classified them based on gender probability using Microsoft’s demographic tool, which will classify a query by it’s probability of being male or female. We looked at it this way: not whether the searcher was male or female but did the particular query fit a gender stereotype—did it have a kind of a male, for example, feel to it or stereotype implications?

Having done previous work with personalization, and gender specificity does fall into a broad category of personalization, Jansen had his own hunches about what he found. As it turned out, his hunches were wrong:

Jansen: The results to me were counterintuitive from what I expected. Usually, the idea of personalization is that the more personalized you get, the higher the payoff, the efficiency and effectiveness is. [But when we looked at the data] in terms of sales, far and away the most profitable were the set of queries that were totally gender-neutral. We took the queries and divided them into seven categories: “very strongly male,” “generally male,” “slightly male,” “gender neutral,” “slightly female,” “strongly female,” “very female.” By two orders of magnitude, the most profitable were the ones that were totally gender-neutral.

Jansen offered examples of “gender neutral” terms:

Jansen: We defined gender-neutral to be were queries that the Microsoft tool classified up to like 59% either side. So we had a fairly big spread here. Here are some examples of queries based off the Microsoft tool: “electronic chess.” The Microsoft tool classified that 100% male. For a gender-neutral query—“atomic desk clock” and “water purifier.”

At this point, the mystery of why “gender neutral” performed at at a significantly higher level remains to be solved, but Jansen has some thoughts:

Jansen: One thing that is coming out in the personalization research is that at a certain level, we have totally unique differences. You can personalize to a general category and to a certain level, but beyond that, it’s either not doing much good or may actually get in the way. And that may be something that is happening here—that these particular, very targeted gender keyword phrases are just not attracting the audience that the more gender-neutral queries and keywords are.

Again, it’s a “why” thing. We spend a lot of time in web search trying to personalize to the individual level and really haven’t got very far. But now people are trying to do things like personalize to the task rather than the individual person, and there’s some interesting things happening there. Spell checks and query reformulations and things like that are very task-oriented rather than individual searcher oriented.

Dr. Jansen’s point about how personalization might be better aimed at the tasks we’re engaged in rather than the people we are led to further speculation about where personalization might take us in the future.

Jansen: [Personalization] is just so hard to do. You know, Gord is different than Jim, and Gord today is different than Gord was five years ago. Personalizing at the individual level is just very difficult and may not even be a fruitful area to pursue.

We’re nonlinear creatures, we’re changing all the time. I can’t even keep up with all my changes and I can’t imagine some technology trying to do it. It just seems an unbelievably challenging, hard task to do.

I brought up the point that even we don’t know why we do the things we do, because so much of our decision making is driven by unconscious factors. It’s a thought that’s crossed Jansen’s mind as well:

Jansen: I’ve commented on that before in terms of recommending a movie or book to me. I don’t even know what books and movies I like until I see them. Sometimes I pick up a book and say, “Oh, I’m going to really love this,” only to get a chapter into it and realize “Okay, this is horrible.” And I think you see that in the NetFlix challenge— So many organizations have labored for a decade now, and finally it looks like perhaps this year someone may win by combing 30 different approaches simultaneously to the very simple problem of “Recommend a movie. It’s just amazing the computational variations that are going on.

From personalization, our conversation then veered to mobile (not such a long detour, really). To me, the intersection of search and functionality has the most potential on our mobile devices. But the need to “get it right” is substantially higher, given the inherent challenges of handheld devices: limited screen real estate and input challenges.

Jansen: Everybody is saying (again), “This is the year mobile searching’s going take off.” It’s been going on for four or five years now, and really, at least here in the US, it hasn’t really happened yet. But what I think is going to make it hit the mainstream is this combination of localized search. When you have a mobile device, the technology has so much more information about you: it’s got your location to within a couple feet, the context that you’re in can really start entering the picture and information gets pushed to you—I’m thinking tagged buildings and restaurants and cultural events and on and on. And so with my mobile device, where I can talk into it, I don’t even have to type anything. I want “what’s going on in the area?” and it automatically knows my location and the time and perhaps something about me and the things that I’ve searched on before. “Oh, you like coffee shops where there’s some music playing. Guess what? Boom. There’s five right near, in your area that have live entertainment right then.” So I think in that respect it’ll be a little more narrowed search, but the technology will have so much more information about us that in a way it makes the job easier. The problem’s going to be the interface and the presentation of the results.

Imagine being able to walk through a town… I live in Charlottesville, Virginia. Tons of history here from 400 years ago when Europeans first settled here, Thomas Jefferson, James Madison, etc., etc. Being able just to walk down Main Street and have tagged buildings interface with my mobile device… I’m a big history buff and so getting that particular information, one, pushed to me or at least available to push when I ask for it is a wonderful, wonderful area of personalization. This idea of localized search and mobile devices and mobile search may be the thing that brings it all together and makes mobile search happen.

Given the direction of conversation I had to ask Jim about the privacy implications of all this functionality. Let’s assume that Google is the likely candidate that assemble this search “utopia.” What price might we have to pay to enable Google’s effectiveness as our own personal digital concierge, or, more sinisterly, our “Big Brother?”

Jansen: You know, the “Big Brother” idea label has certain negative connotations, so I don’t want to say that Google is Big Brother-ish in that regard. But certainly I think with their movement into free voice and free directory assistance, they will soon have a voice data archive that will allow them to do some amazing things with voice search, which would be an awesome feature for mobile devices. Being able to talk into a mobile device, have it recognize you nearly 100% of the time and execute the search.

My final question for Jim was how much of a priority should Google make innovation in the mobile search space:

Jansen: Google of course is the one that knows what they’re doing, but certainly I think it would be naive not to be exploring that particular area. And I think the contrast from what you said about Microsoft and the desktop, the desktop is just so busy. You’re getting so many different signals in terms of business, personal things, my kids use my computer sometimes. And so the context is so large on the desktop, but the mobile device, it’s narrower. You know, you have some telephone calls, you can do some GPS things, so the context is narrower but very, very rich in that very narrow domain. I think it’s a really hot area of search.

The entire interview transcript has been posted to my blog. As always, a conversation with Jim Jansen never fails to be interesting.

Jim Jansen is one of the few academics I know that is fascinated with Internet search. He has spent a good part of the last decade looking at patterns in search query and website logs, dissecting them and continually looking for significant trends. Jim and I crossed paths a number of years ago and have kept in touch ever since. When Chris Sherman asked someone to talk to Jim about his latest research, I was quick to volunteer. As I said in my last column, a chat with Jim is always fascinating.

First, a quick word about Jim’s background. Dr. Jansen is an associate professor in the College of Information Sciences and Technology at Pennsylvania State University. He has more than 150 publications to his credit. In fact, in the time I’ve know Jim, he has turned out papers at an amazing rate. He’s also co-author of the book, Web Search: Public Searching of the Web and co-editor of Handbook of Weblog Analysis. Jim moved into his academic career from the military, where he taught at West Point.

Jim’s research project

Jim has spent the last few years working on massive data sets that have been made available by large, high traffic sites. He has taken a statistical approach to dissecting these data trails and gaining insight into behavior through that analysis. You could say that Jansen is investigating what John Battelle has called the database of intentions.

The database of intentions was a concept Battelle introduced in his book The Search, How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture. In his words, the database of intentions is:

The aggregate results of every search ever entered, every result list ever tendered, and every path taken as a result. It lives in many places, but three or four places in particular hold a massive amount of this data (ie MSN, Google, and Yahoo). This information represents, in aggregate form, a place holder for the intentions of humankind – a massive database of desires, needs, wants, and likes that can be discovered, subpoenaed, archived, tracked, and exploited to all sorts of ends. Such a beast has never before existed in the history of culture, but is almost guaranteed to grow exponentially from this day forward. This artifact can tell us extraordinary things about who we are and what we want as a culture. And it has the potential to be abused in equally extraordinary fashion.

As Battelle admitted, this was a BIG IDEA. Jansen has taken a methodical approach to slicing and dicing his own subsets of the database of intentions. I’ll be talking to Jim about one of these slices in today’s column, but first, Jim will explain a little more about his current project:

Jansen: I have several research projects going on. One that I really find interesting is analyzing a five calendar year search engine marketing campaign from a major online retailer and brick-and-mortar retailer. It’s about 7 million interactions over that time, multi-million dollar accounts and sale. A fascinating temporal analysis of a search engine marketing effort. I’ve been looking at that at several different levels—the buying funnel being one, aspects of branding being another, and then the aspect of some type of personalization, specifically along gender issues.

The buying funnel and search engine strategies

The slice of Jim’s research I want to focus on in this column is the idea of a search “buying funnel.”

Marketers have long cherished the buying funnel model. The foundations of this model go back to the AIDA model—Attention/Interest/Desire/Action—introduced by Elias St. Elmo Lewis in 1898. The roots of this model run deep. The labels of the stages of the funnel vary somewhat, but generally they align with Need—Awareness—Consideration—Purchase. In my own recent research in the B2B marketplace, I have expressed some doubt about the applicability of the funnel as a workable model. I don’t dispute the stages, but I do question the idea of a linear “funnel,” with prospects moving obediently from one stage to the next. Based on my observations, many purchases are just not that simple.

But if we accept that some sort of buying funnel is in place, and we know (because we do know) that search is used to qualify and research buying decisions, than it makes sense that there should be a corresponding search funnel. Jansen went in with this basic hypothesis:

Jansen: One goal was to verify whether the buying funnel was really a workable model for online e-commerce searching or was it just a paradigm for advertisers to get their handle around this chaos. And the other goal was to discover if it’s an effective model, what can it tell us in terms of how advertisers should respond?

Jansen’s not the first to explore the territory of a search funnel. A comScore study of search behavior in consumer electronics in 2004 questioned the existence of a search funnel:

“The results of the study challenge a widely held belief that most consumers begin the product search process by using a generic search term (e.g. “plasma TV”) and then later refine their search activity to product-specific terms (e.g. “Sony Plasma KE-42M1”). Operating under this assumption, many retailers and manufacturers believed that investing only in product-specific terms allowed them to reach the majority of in-market consumers closer to their purchase decision. In reality, by taking this approach, marketers are missing the vast majority of their addressable market, since most consumers never use these types of terms.”

So, what did Jansen find in his data set?

Jansen: In terms of the first question, we had some mixed results. One, at the individual query level you can classify individual queries into different levels of this buying funnel model. There are unique characteristics that correspond very nicely to each of those levels. So in that respect, I think the model is valid.

Where it may not be valid is specifying this process that online consumers go through. We found that, no, it didn’t happen like that. There was a lot of drop-out and they would do a very broad query and then there may or may not be more specific queries after.

So we looked at the academic literature—what theoretically could deal with that or explain that?—and the idea of sufficing seemed to fit. If it is a low cost, they won’t spend a lot of time searching… they will just purchase it and buy it.

In terms of classifying queries in terms of what advertisers’ payoff is, I think the most interesting finding was that the purchase queries, the last stage of the buying funnel, were the most expensive and had no higher payoff than the awareness or the very broad, relatively cheaper queries. From talking to practitioners, that is a phenomena that they have noted also… which is why a lot of people bid still on very broad terms, to snatch these potential customers at an early stage.

Jansen’s findings seemed to support the earlier comScore findings and showed that search activity, just like consumer activity, doesn’t go in predictable or logical straight lines. I dove deeper on this particular area with Jim:

Hotchkiss: We similarly have found that you can’t assume a search funnel is happening because people use search at different stages and they’ll come in and then they’ll drop out of the process, and they may come in later or they may not, they may pursue other channels. But the other thing we found is sometimes there’s a remarkable consistency in the query used all the way through the process and we that quite often can be a navigational behavior. It can be people who say, “Okay, the last time I did this, I searched on Google for so-and-so and I remember the site I found was the third or fourth listing down,” and they use the same route to navigate the online space over and over again. So if you’re looking at it from a pure query level, it’s a bit of a head-scratcher because you’re saying, “Why did they use the same query over and over?” Again, it’s one of those nuances of online behavior. Did that seem to be one of the possible factors of some of the anomalies in the data?

Jansen: Well, that trend or something similar to it has been appearing in a lot of different domains and researchers are kind of attributing it to “When I do a query, I expect a certain result.” So with a query that may be very informational, what we’re finding is that searchers expect a Wikipedia entry… a very navigational intent behind that very informational query. And I think the phenomena you’re describing is very similar. We have a transactional-type query and users are expecting a certain web page, a navigational aspect: “Okay, I have an anchor point here that I’m going to go to.” I just looked at a query log from a major search engine and an unbelievable amount of queries were navigational in nature.

The other area I found fascinating was this concept of “sufficing,” or, as Herbert Simon labelled it, “satisficing.” Jim and I used that as the jumping off point for a rather interesting discussion about satisficing and how it might play out on a search engine. I’ll just let the conversation play out here, as I think it’s self-explanatory.

Hotchkiss: You know, the idea of satisficing, of taking a heuristic shortcut with their level of research, is also interesting. It seems that if the risk is fairly low, the online paths are shorter. Is that what you were finding?

Jansen: Yes… the principle of least effort is how it’s also presented. We see it in web searching itself in how people interact with search engines and how they interact with sites on the web. They may not get an optimal solution, but if it’s something that’s reasonable and it’s good enough, they’ll go for it. And that seems to be occurring in the e-commerce area also. “I want to buy something relatively cheap. Okay, this particular vendor may not have the best price, but guess what? It’s close to what I’m thinking it should be. Just go and get it done, get it over with, buy it.”

Hotchkiss: And I would suspect that that would also be true in product categories where you mentally have a good idea of what an acceptable price range would be, right?

Jansen: Yes.

Hotchkiss: So if it’s a question of making a trade-off for $2 but saving yourself a half hour of time, as long as you’re aware of what those price ranges would be, you’re more apt to take that shortcut, right?

Jansen: Yes. It does assume some knowledge and risk mitigation—if it’s a small purchase. That varies a little bit for each of us, but you’re willing to cut your costs of searching and trying to find the best deal just to get it done.

Hotchkiss: Part of this too would be your level of personal engagement with the product category you’re shopping in. For instance, I’ll spend way too much time researching a purchase of a new gadget or something that I’m interested in just because I have that level of engagement. But if it’s basically a purchase that’s on my to-do list, if it’s just one task I have to get done and then move on to the next thing, I suspect that that’s where that satisficing behavior would be more common.

Jansen: Now you bring up a really good point. If it becomes entertainment— like a gadget that you enjoy researching—it’s no longer work, it’s no longer something you get done. The process of doing it makes it enjoyable so you don’t mind spending a lot of time. In those kind of cases, the goal really is not the purchase, the goal is the looking.

Hotchkiss: And we found that alters the behavior on the search page as well. If it’s a task-type purchase where I just have to go and get there, you see that satisficing play out on the search page too. Typically when we look at engagement with the search page, you see people scan the top four, three or four listings. It’s that satisficing type of intent where you say, “I just want to buy this thing.” What you’ll see is that people scan those first three or four and pick what they feel is, like you say, the path of least effort. They go down and say, “Okay… it’s a book. Amazon’s there. I know Amazon’s price. I’m just going to click through and order this,” but if it’s entertainment, then suddenly they start treating the search page more like a catalog where they’re paying more attention to the brands and they’re just… they’re using that as a navigational hub to branch off to three or four different sites. And again, it can really impact the nature of engagement with the website… or with the search page.

Jansen: Absolutely, and I really like your analogy of a catalog. You know, there are some people that love just looking at a catalog—flipping through it, looking at the dresses and shirts or gadgets or sporting gear or whatever. And that’s a much different engagement than flipping through the classified ads trying to find some practical thing you need. The whole level of engagement is at totally opposite ends of the spectrum, really.

Hotchkiss: As an extreme example of that, we did some eye-tracking with Chinese search engines and what we found was, with Baidu in particular, people using it to look for MP3 files to download. So when we first saw the heat maps—and of course it was all in Chinese, so I couldn’t understand what the content on the page was without having it translated—but I saw these heat maps going way deeper and much longer than we ever saw in typical North American behavior. We saw a level of engagement unlike anything we had ever seen before. And I said, “Well, what’s going on here?” and that was exactly it. It was a free task… they were looking for MP3 files to download and they were treating the search page like a catalog of MP3 files. So they were reading everything on the page and I think that’s just one extreme example of this catalog browsing behavior that we were talking about.

So let’s go to one of the other findings on the buying funnel which was that quite often the more general, broader categories from an ROI perspective can perform just as well as what traditional wisdom tells us is your higher return terms which are closer to the end of the funnel—they’re more specific, they’re longer, they’re more transactionally oriented. What’s behind that?

Jansen: Well, in a lot of these questions there’s no simple answer… there’s plenty exceptions to the rule of what you have just described there. There are some very broad terms that are very cheap, others that are very expensive. On the purchase side, there are some key phrases that are very cheap because they’re so focused and others are expensive. But in this particular analysis—and again, this was 7 million transactions over 33 months, from mid-2005 to mid-2008—the awareness terms were cheaper than the purchase terms and they generated just as much revenue.

I think a lot of it is that perhaps the items this particular retailer were selling fell into that sufficing behavior where they were gifts, fairly low-cost items. There was just no need to progress all the way to that particular purchase phase.

Really… it was very unexpected. I really expected those purchase terms to actually be cheaper because they were more narrowly focused and to generate more revenue, but overall… it didn’t turn out that way.

Hotchkiss: That brings up an interesting point that we’ve seen with client behavior, especially given the current economic condition. What we found is a lot of clients are tending to optimize down the funnel—they are looking at their keyword lists they’re bidding on and move further and further down to more and more specific phrases, because the theory is—and generally they do have analytics to back this up—that there’s greater ROI on that because these are usually people that are searching for a specific model or something which is a pretty good indicator that they’re close to purchase. But I think one of the by-products of that is that as people optimize their campaigns, those long tail phrases are getting more and more expensive because there’s more and more competition around them, and as people move some of their keyword baskets away from those awareness terms, maybe the prices on that, it all being based on an auction model, are starting to drop. Do you think that could be one of the factors happening here?

Jansen: That very well could be. The whole online auction is designed around [the idea that] as competition increases, cost-per-clicks will increase also. It also may be that those particular customers don’t mind clicking on a few links to do some comparison shopping and may end up going somewhere else… they may have a higher intent to purchase, but the competition among where they’re going to buy is more intense.

As always, I found the conversation with Jim enjoyable and enlightening. We continued to talk more about branding and search and the implications of personalizing to gender, both areas that Jim explored in this latest study. I’ll cover that territory in my next Just Behave column.

This past week, I was fortunate to be able to interview Penn State professor Jim Jansen about some of the work he’s been doing. I had hoped to get the interview transcript back in time for this column, but unfortunately the timing didn’t work with the deadlines (that will be the next column). My conversation with Jim did bring something to light, however, that I would like to cover in this week’s Just Behave.

Quant & Qual: the yin and yang of research

Jim works in the quantitative world. For his most recent work, he’s been digging into a massive dataset that includes the click stream of millions of visitors to one large commercial site. Slicing and dicing data is Jim’s thing. He’s one of the best in the world at taking these massive sets of digital footprints and boiling it down to discover the interesting common threads.

I, on the other hand, am much more comfortable in the qualitative world. I like knowing what makes humans tick. The research I love to conduct depends more on a handful of people than huge mounds of data. Observed behavior and signals of neural activity continually fascinate me.

Jim and I have had a correspondence going back several years now, punctuated by one single face-to-face visit. And although we approach questions from different perspectives, we both enjoy the times we get to bounce our respective view of search behavior off each other. Jim provides me with a meticulously crafted view of “what” happens and I attempt to provide some educated guesses as to “why” that might be. I have had similar conversations with the data analysts at comScore, Compete and Google.

Is “maybe” an answer?

A month or so ago, I was talking to a reporter who was looking at the evolution of market research and we got onto the question of the qualitative/quantitative balance. She asked me why there weren’t more psychologists doing market research? My belief was that psychologists don’t provide clear-cut answers. Everything is a qualified “maybe.” And in this age of digital accountability, a “maybe” just doesn’t cut it if millions or billions are at stake.

However, I do think the balance between qualitative and quantitative is unfairly skewed towards logic and hard data. The reason I enjoy my conversations with Dr. Jansen so much is that we have a very healthy respect for each other’s approach. We know that both approaches have their place and need to work together. To understand “how”, you need to know both “what” and “why.”

We’re short on the “why”

Today, there are no shortage of places marketers can turn for answers to “what.” There are site analytics and panel based tools such as comScore, Hitwise and Compete, as well as Google’s own tools. In short order, a marketer can become buried in a mound of data, endlessly analyzing the “what.”  But even with all the information available, this view can be of limited value if you have never think about “why.”

So, where do marketers turn for “why?” For example, at Enquiro, we recently wrapped up an extensive research project looking at organizational buying behavior. We wanted to know “why” businesses buy from certain vendors and not others in the new digital marketplace. We started by looking for existing research into this and came up dry. If you’re looking for why, there are not a lot of resources available to you. When we started looking, our most promising trails led back to our own research—gratifying for our egos but not very satisfying for our curiosity.

Search is a connection

One of the things about search activity that quickly became apparent to me is that you can’t analyze it in isolation. Search is a connection between a person’s intent and the most relevant information. Looking at search activity has less value if you don’t know the intent that lead to that activity. You look at faceless, intentless aggregated data and try to determine patterns. It hearkens back to B.F. Skinner’s “black box” approach to psychology. Jansen has demonstrated that this approach can yield interesting findings, but they become all the more interesting when you combine that with pure qualitative exploration of people’s motives and objectives. That gives you the framework that helps make sense of the quantitative data. Yes, accepting qualitative research requires a leap of faith, but the best qualitative researchers thrive on gut instinct. It’s that quality that makes them such a rare breed.

The time is “right”

The book I’m reading right now is Daniel Pink’s “A Whole New Mind“. In it, he compares left and right brain thinking, a convenient metaphor for quantitative, rational thinking and qualitative, creative thinking. Left brain thinking focuses on details, facts and figures. Right brain thinking synthesizes all this into a bigger picture, using intuition to fill in the inevitable gaps. Pink maintains that our left dominated world will veer more to the right in the future.  The time for right brain thinking has arrived.

I hope the same is true for market research. I think we need more of the conversations I’ve so enjoyed with Dr. Jansen, a convergence between right and left approaches to understanding search marketing. We need both for a complete picture of human behavior.

The deal is done. Microsoft has swallowed Yahoo search whole and we can all be put out of our long, lingering misery. Yahoo has given up on search and thrown in the towel. But, outside this industry and our incestuous little gossip circle, what does it really mean for average folks? Does it make a difference… really? When all is said and done, will the news amount to a hill of beans?

Well, in the long run, it’s probably a good thing for most of the players involved, including users. My original concern when rumors of Microhoo started to fly was the distraction that the integration of two very different and somewhat challenging cultures would become. I was afraid that the search user experience would get lost in the process. But the revenue share split does away with that concern, which is good for users. Yahoo can continue to be Yahoo, minus the “distraction” of search. Microsoft can get serious about search with a more viable economic engine to justify their investment. Basically, the deal gives Microsoft a whole lot more eyeballs for Bing, and that’s going to be beneficial for everyone.

But there are some other reasons why the search user could benefit from this deal:

A release of talent

There’s a whole bunch of PO’d, disenfranchised search engineers that have just had a frightening but potentially liberating view of the future: freed from the shackles of the rapidly sinking S.S. Yahoo, they might actually have a chance to do something meaningful in search.

I’ve had the opportunity to talk to various members of the Yahoo! search team over the last several years, under Semel, Yang and Bartz. The underlying current was always one of thinly veiled frustration. They deserved better. They were smart and, once upon a time, passionate about search. But for the past several years they were left hanging out there, blowing in the breeze. They yearned to do something meaningful. Now, with the ladder kicked out from underneath them, they have no alternative. Whoever hasn’t already jumped ship will be forced to find new, and hopefully, more rewarding homes.

Pushing the wolves back from the door

At the risk of sounding like a broken record, or, less anachronistically, a skipping MP3 file, search needs a major shot of innovation. The dysfunctional competitive environment that we’ve had for the last few years put all the major players in a position where innovation wasn’t encourage. Specifically:

Google. As the market leader with a major marketshare, almost completely dependent on the search advertising revenue stream, it’s hard to rock that boat too vigorously. Every minor change in the search UI could have a potentially negative impact on ad clickthroughs. The dynamics of the SERP are subtle and small changes can have big impacts on behavior. Given the obsessive focus on quarterly revenue numbers by Wall Street analysts, and the macro economic impact of the recession, this was not the time to boldly screw around with your golden egg-laying goose. Google always speaks a good game when it comes to hyping their efforts to innovate in search, but come on, how different is the Google SERP now from what it was 4 years ago? How innovative have they been with AdWords?

Microsoft. Microsoft was the one who was really in a position to innovate, but despite repeated commitments to search from everyone from Gates on down for the last several years, the company’s track record has been less than stellar, to be exceedingly kind. Bing is the first sign that they’re motivated to get it right, and that’s really more of a catch up move than anything. But at least they’re in the game now. I think the biggest problem was getting alignment in the gargantuan, lumbering Microsoft corporate structure. There was a lot of left hand/right hand lack of communication from Spanky and the gang in Redmond. It’s hard to innovate when you don’t even know who’s working on what.

With Bing, Microsoft showed they can execute. Now, let’s see if they can innovate.

Yahoo. You don’t innovate on the Titanic: you just try to survive. Nuff said.

What the Microsoft-Yahoo deal does is allow Microsoft’s search champions to better sell the need to innovate within the company. They have a bigger market share, can forecast significant revenue potential and make the case for investing in rapidly upping the search experience bar.  Microsoft can afford the fluctuations that are inevitable in ad revenue as they experiment with the UI, because search is a pretty small drop in the revenue bucket. And it’s certainly not that they don’t have the resources—all they need is the corporate prioritization and alignment. This deal may just do that for them.

A bigger ad inventory

Microsoft CEO Steve Ballmer sent the signal of the true value of a Yahoo partnership when he said last December:

“We’re fully prepared to compete without any partnership with Yahoo. We don’t need to act. Would it be advantageous for both of us to make a deal? Look, the fundamental basis for doing the search deal with Yahoo has to do with critical mass in the advertising marketplace. It doesn’t have to do with technology, or any of these other things, it really is a market phenomenon. Together we would have more advertisers… which means we’d have more relevant ads on our page. We’d have higher monetization levels possible in front of us because there would be more people bidding on more key words. Most importantly, Google would have perhaps a real credible competitor sooner.”

Success in search is all about relevancy. And if ads are going to be part of the results set, those ads better be as relevant as possible. The bigger the ad inventory, the more relevant they will be.  It’s a pretty simple equation.

And ad relevance is massively important to the user experience. At Enquiro we’ve done several studies that either directly or indirectly examined the importance of ad relevance to the overall search experience. The results are abundantly clear: the better the ads, the happier the user. The deal gives Microsoft a massive increase in advertiser inventory, and that’s perhaps the most important asset in the deal. It doesn’t matter that Yahoo gets 88% of the revenue. That’s short term thinking.  This is about running all those ads through the Microsoft platform, giving them the ability to control quality, targeting and relevance. The goal is gaining market share, and you need the highest quality ads possible to do that.

A stronger commitment from advertisers

No longer is an ad buy on Microsoft tossing them a bone out of guilt or a gesture of protest against a Google monopoly. With a combined market share pushing 30 points, that’s a serious slice of available eyeballs. Microsoft just became a mandatory buy. That allows them to build stronger relationships with advertisers, getting a more serious commitment in return. If Microsoft can pull the pieces of AdCenter together like they did with Bing, they’ll have a pretty powerful ad management platform. And Microsoft doesn’t suffer from the same passive-aggressive relationship with marketers that still lingers in the halls of the Googleplex. Advertisers look at Google with a mixture of resignation, fear and resentment. More than a few will willing fall into Microsoft’s embrace, now that there’s a justifiable reason to do so.

How does this help the user? Happy advertisers mean happy corporate execs up in Redmond, which means more of an appetite for innovation. Expect Microsoft to be more innovative in offering sponsored search formats. And happy advertisers also means more relevant ads (see my previous point).

More serious competition for Google

This deal sends a signal Google can’t ignore. They’re still in the search driver’s seat, but at least now they can see someone in their rearview mirror. If Microsoft can adopt a passion for innovation and push the envelope, Google will have to respond in kind. The search experience will evolve more rapidly, hopefully kicked out of the revenue obsessed stasis that it’s currently in. Stagnation benefits no one except the analysts and bean counters who insist that quarter over quarter performance is the only metric that matters. We’re way too early in the game to be that cautious and boring.

Will the Microsoft-Yahoo deal break the Google habit? No. In fact, Google will probably net a couple more percentage points out of this in the short term. But this lays the foundation of a more competitive market place, which can’t help but benefit users.

The deal puts Microsoft in the game. Now, let’s see what they do with the opportunity.

In my last Just Behave, I talked about how the vast majority of search engine users never go beyond the vanilla functionality of a search engine. They skip along the surface of search, never diving deep into advanced queries, filters or clicking on the tabs and links behind which lies some truly impressive capabilities. This paradox had two sides: why don’t we work harder at search, and; if we never explore all that rich functionality, why do search engines keep developing it? Why did Google create Wonder Wheel, why does Bing offer several layers of search refinements and why does Yahoo still have a sandbox? What’s the point of all this if we never use them?

Search engines keep testing the development waters because they have too. Everyone acknowledges that we’re still in the very earliest stages of digital information searching. If we plotted this on an evolutionary time scale, we’d be just emerging from the primordial ocean. And no one is sure what search advancement might be the one that tips the balance and creates a significantly more evolved experience for the user.

Revolution through evolution

I’ve said on multiple occasions that Bing is an evolutionary advancement in search, not a revolutionary one. Personally, I have no problem with that. I believe all advancement is evolutionary in nature. Revolutionary change is built on the back of hundreds of small evolutionary steps forward. Sometimes, one of those steps creates a tipping point and everything suddenly shifts dramatically.

Changes are changes: some cause nary a ripple in the vast pool of natural selection, and some cause a species to start walking upright. The problem is that you’re never sure which one is going to be which. The problem I have with Bing has nothing to do with product development and everything to do with product marketing. The Bing advertising campaign is promising something the search engine isn’t ready to deliver. Not yet, anyway.

So the engines have to innovate and develop new functionality, even if nobody is ready to use it yet. Because some of that functionality will form the foundation that the new generation of search will be built upon.

The iPhone: revolutionary or evolutionary?

Everyone looks at the iPhone as a revolutionary product. But there’s nothing in the iPhone that didn’t exist in some form before. Multi-touch displays? Nimish Mehta developed the first example at the University of Toronto in 1982, 25 years before the iPhone debuted. The iPhone’s ability to detect motion? The electronic brain behind that “revolutionary” advancement is an accelerometer, technology that is decades old.

There is no single revolutionary thing about the iPhone. What is revolutionary is how all these evolutionary advancements came together. The same will be true when Search breaks its current paradigm. Suddenly the world will discover amazing new functionality that’s been around for years, hidden behind an unused tab or hidden hyperlink.

Functionality designed for a new interface

What is guaranteed to change is the way we interact with search. Search needs to become much more intuitive and deliver more relevance in less real estate. Up to now, search engines have had the luxury of delivering a results “buffet”, 20 to 25 links spread over a big desktop display. It leaves it to the user to pick and choose the best link, which hopefully the engine has placed to the upper left.

In the future, engines will have to “get it right” more often, presenting the best result at the top of the page, so when we access from a mobile device, we will see what we’re looking for in a much smaller display. Don’t give me a buffet and ask me to choose. Deliver me exactly what I’m craving right now, even if I’m not sure what to ask for.

We also need to do more with search results. It’s not enough for them to be relevant. They also have to be useful. Anticipate what I want the information for and then take me several steps down that path without me having to do anything. Mash up results with other applications and bring my ultimate objective several clicks closer.

In summary, to revolutionize search, we need to do three things:

• Get it right 99.9% of the time in as little real estate as possible
• Deliver those results with as little explicit user input as possible
• Anticipate what the user is going to do with the results and take them as far down that path as possible

All the things the engines are testing right now is hopefully getting us closer to one or more of those three objectives. And while individually those advancements aren’t enough to get us to play with the newest beta in Yahoo’s Sandbox or Google’s Lab, in combination they might significantly change the game. Going from ho-hum to revolutionary is sometimes not that big a leap. It’s simply a small evolutionary step in the right direction. Keep in mind, humans share 96% of our genetic code with chimpanzees. It’s the 4% difference that counts.

Do you regularly use Google’s Wonder Wheel? How about SearchWiki, Yahoo Correlator or Search Monkey? I’m guessing about 95% of you said no. And you’re no ordinary group of searchers . In terms of search literacy, you’d be ranking in the top 0.1% of the population. In a randomly assembled group of 1000 people, you’d stand out as the expert on web search. It’s the reason you actually take time out of your day to read a column on search behavior. Believe me, normal people don’t read this stuff.

Even Geeks don’t tweak search

I like all of the above search refinements. They make the user experience better. And when it comes to being a search behavior geek, I’m even geekier than you.  But I have to confess something. I don’t use any of them. Nary a filter, an advanced query, nor a Boolean operator can be seen in my search history. My searches are plain vanilla, boring, ridiculously simple. The way I use Google is not so different than the way my mother uses it, and she just got her first computer two years ago. Her definition of advanced video gaming is Freecell.

Google just introduced a whole new batch of search functionality, including Wonder Wheel. Together, they can dramatically increase the power of Google to help you narrow in on exactly what you’re looking for. Yet all this functionality will never be used by the vast majority of users. The same is true for the advanced customization available on Yahoo! Expect similar filtering and functionality on the new flavor of search coming soon from Microsoft. So, the question is, if we never use it, why do they build it?

Googling Google on Google

Let’s start with the first part: Why do we never use it? It’s a question with two possible answers: digital literacy and digital diligence. Let’s talk literacy first.

People in the search marketing industry constantly make fun of all those stupid users who search for search engines on search engines. I’ve been in the business for well over a decade now and it’s a running joke: “Look at all those helpless fools who Google “Google” on Google.” Guess what? It’s no joke. It’s the way people navigate the web. It’s not that they’re brain dead. It’s that they’re brain “efficient.” They’re operating on autopilot, saving the intellectual horsepower for more important tasks. It’s only now that Google diligently captures every search and reports on our aggregated activity that we realize how silly our online navigation habits look in retrospect.

I never realized how reliant the average web user was on Google for navigation until I built a website for my daughter’s elementary school. On the day I pushed the site live, I was going back and forth with the principal to make some last minute changes. I had just published the site and checked to make sure it was available online. When all looked good, I sent a quick email to the principal to let her know it was live. A few minutes later I got an email from her saying she couldn’t get to the site. I checked again. Everything looked fine on my end. Thinking she might have to clear her cache to get rid of the under construction page and knowing that she was relatively new to online, I got on the phone to walk her through it. It took about 5 minutes before I realized what the problem was. She was trying to navigate to it through Google. She knew no other way to get to sites. The site was 20 minutes old. Google’s efficient, and I know a little bit about site optimization, but neither of us was that good. St. Joseph’s Elementary was non-existent as far as Google was concerned, and that was the only path the principal knew to take.

Navigating by search

I thought this was isolated, but one week later I was talking to my Mom. Turns out she navigated the same way. So did my step dad. And my aunt. I started asking around. Turns out most people I know navigate the web through search. So we started asking that question in surveys. It appears my circle of family and friends weren’t unique in that. Here’s one recent example. In a recent survey of B2B buyers, we asked them how they would get to a site they already knew. 55% said they would get there through search, either by going directly to a search engine or through a tool bar. I suspect there’s a self reported bias there: I think the number’s even higher. This is huge for marketers. If I’m ordering copy paper that I always order through Staples.com, chances are better than 50% that I’ll get there through a search engine.

The vast majority of people who use search do so because it’s easy.  Google and competitors have managed to provide the easiest path to most things we’re looking for. And because they’ve done that well, everyone knows Google. But unless you’re in the industry, you don’t spend a lot of time wondering about advanced search options. It’s not a topic of conversation at the office. My teenage daughter is not burrowed away in her room, texting all her friends about Google’s Wonder Wheel. Fact is, almost everyone else in the world looks at search the same way they look at electricity: they don’t care what makes it work, but it damn well better work when they need it.

Don’t expect us to think while we search

For Google, there’s a good new/bad news side to this. Searching is a habit. We don’t think, we just “query” in the nearest available box. This translates into massive search volume. That’s the good news. That’s also the bad news. We don’t think. Which means all that lovely functionality the engines work so hard on will never be used, because it represents a detour on the quickest path to our objective. Google knows this. That’s why all that wonderful functionality is hidden behind a small blue “options” link tucked away in the upper left corner, well out of the visual path of 99.9 percent of search users.

And that brings us to digital diligence. We will only work as hard as we have to to find information, Peter Pirolli’s Information Foraging Theory says we use the same equation to track down information that we use to track down food: The investment should always be less than the reward. If we burned more calories finding food than we received from the food, we wouldn’t last long. There are no calories involved in information, and we burn precious few using Google, but the principle is still the same. Search is a utility, not something we invest time in just for the hell of it. As fun as Search Monkey or Wonder Wheel might be for us search geeks, it’s not something the 99.9 % of “normal” people would ever use. They’re too busy doing “normal” things, such as watching Dancing with the Stars.

So what about searching for serendipity, the example that commonly gets brought up every time I talk about the utility of search? What about using search just to explore the online world, open to discovering whatever we might find? Wonder Wheel’s graphic interface based on semantic clustering would be just the ticket for that, wouldn’t it?

You know, I use search a lot. I use a search engine dozens of times per day, on average. And in a month, I doubt I launch even one serendipitous search. I just don’t explore the web that way. The odd time an interesting listing I didn’t expect might distract me, but I seldom start out with that intent. I might cast the search net wide, or I might be specific and navigational, but in both cases, Google does a pretty good job most times. And based on the rule of marginal value, pretty good is good enough to keep me coming back to the Google “patch” and not worry about finding something better.

Despite all this, I don’t think search is solved yet. I agree with the common wisdom that we’re still early in the game and search has a long way to go. Both the front end of search that we touch and the functionality that lies behind have barely begun to evolve. But that evolution depends on giving us better results without asking us to do anything more as users. No more filters, no more clicks, no more advanced search options. There will always be those searchers that appreciate that, but in this normal distribution curve, they’re definitely the outliers.

Which brings us back to the original question. If the vast majority of us don’t use the extra functionality, why do the engines keep building it? Because, just like everything, search is a Darwinian game. And next time we meet here in Just Behave land, you 0.1 percent of the population who actually care about such things, that’s what we’ll be talking about.

My daughters are different than I am. And not in just the obvious ways: like age and gender. They’re different in the way they use technology. The reason why lies in the way the brain is formed. In today’s Just Behave, I’d like to explore what might be happening to our children’s brains.

Several months ago, journalist Nicholas Carr asked the question: Is Google making Us Stupid? In past Just Behave columns, I’ve explored the impact of Googling and Internet exposure on our brain, including a fascinating study conducted at UCLA’s Semel Institute with Drs. Gary Small, Susan Bookheimer and Teena Moody. I had the opportunity to chat with Dr. Moody about the study. One of the things that emerged was that it appears that our brain literally rewires itself as we become more familiar with technology. There’s nothing particularly startling in this. This neuronal plasticity is the mechanism of learning. But not all brains have equal amounts of plasticity. Generally, the younger the brain, the more plastic it is.  So, one wonders, what happens when you grow up with digital technology?

The building of young minds

As we grow from infancy, there are two periods where the brain is particularly malleable: the first 3 years of our lives, and the years of adolescence. During these times, our brains are literally building themselves from inside so they’re equipped to handle life as an adult. We are shaped by what we do during these times. It’s no coincidence that those are also the most challenging times for parents: the Terrible Two’s and the equally daunting teenage years. As our brains go through wholesale rewiring, we push against the constraints of our environment, including the boundaries set by our parents. It’s all part of growing up.

The process is called pruning. Here’s an urban myth that helps explain it, courtesy John Medina’s book, Brain Rules: A college had completely redone their grounds over the summer, with lush, manicured lawns, gardens and fountains. There was only one thing missing from the design: pathways. As September approached, the construction company was pushing the college president for his plans on where the cement pathways would go.  Knowing the pathways would be permanent, he kept delaying. Finally, as the fall term was imminent, he said to the construction team, “Come back next year. I’ll give you the plans then.” The workers were taken aback. No pathways? But the president wouldn’t budge. Disgruntled, they packed up and left for the year.

Over the next year, students began cutting across the grounds to get from class to class. Over time, the most efficient pathways were defined by thousands of feet, and large islands of untouched green grass also emerged. The most used paths soon became clear. The next fall, the workers returned to get the plan from the college president. “There”, he said, pointing out the window at the network of pathways, “there’s your plan. Put the pathways where people walk.”

As we grow, we keep the pathways we use all the time and the ones we don’t grow fainter through disuse. The brain “prunes” away these pathways, concentrating on strengthening the ones we use more often. We focus on the mental skills most important for survival.

A monkey by any other name

One of those skills is the ability to recognize faces. Humans can recognize and distinguish thousands of faces, which is remarkable, when you think about it. But if we were shown pictures of 12 different chimpanzees, or sheep, or lemurs, most of us couldn’t tell one from the other. They would all look alike. Is this because humans are especially distinctive in their physical features? No, it’s just that our brains have been optimized to tell our Uncle Joe from Paris Hilton. It’s rather important to our ability to function in the world.  But what about babies? This was the question Olivier Pascalis at the University of Sheffield set out to explore.

Dr. Pascalis found something remarkable.  Up until 6 months of age, babies could distinguish the faces of different monkeys. By nine months of age, babies lost this ability unless they had ongoing training in distinguishing one monkey from another. If they did, they retained the ability. In normal development, the time when babies start to be able to distinguish the faces of people close to them falls into this same time period. As we gain the ability to recognize human faces, we lose the ability to tell one monkey from another, unless we specifically work to retain this ability through ongoing stimulation. Like the pathways in the college common, if we don’t use these abilities, they simply disappear.

Digital Fluency

So, if our brains are creating pathways, what happens when we throw new environmental factors into the mix? What happens when we expose our children to something we weren’t exposed to during this same pruning period? They simply become better at it. And they do it in a way that we can never duplicate.

If you learn a language as a child, you learn it as a native. You develop a fluency that you’ll never achieve if you try to learn the same language as an adult.  My wife was born in Canada, but her parents were both born in Italy. Her father immigrated at the age of 18, and her mother at the age of 10. My wife learned English from birth. Her mother learned English as a teen. Her father learned English as an adult. The difference in fluency is noticeable. My wife, of course, is fluent. English is her native tongue. Her mother has a very faint accent, but she is more comfortable in English than Italian. Her father still struggles with some of the idiosyncrasies of the English language and has a noticeable accent, despite the fact that it’s been the language he uses every day for almost 50 years. And that will never change.  These differences have been hardwired into their respective brains through pruning.

Just like my in-laws exposure to English, new generations are being exposed to a totally new language from birth that we had to learn as adults: the language of digital technology.  My daughters learned it from birth. They’re digital natives. And I, even though I spend several hours a day on a computer, will always be a digital immigrant. It’s a profound and fundamental change that we’re just beginning to see the impact of.

iBrain: Digitally Designed

It’s this generation divide in digital fluency that Dr. Moody’s co-researcher, Dr. Gary Small, examined in his book iBrain. As with any attempt to over generalize human populations, the divide between digital natives and digital immigrants is not so neatly drawn. The normal distribution curve that’s ubiquitous in human populations raises its head here as well.  Within digital natives, there are outliers on both ends of the curve. There are those that are addicted to digital interfaces, whether it be mobile texting, video games or YouTube. And there are Luddites here as well. Similarly, there are remarkably wired seniors as well as those baby boomer age and older hanging onto their non-digital worlds by their fingernails. Indeed, one of the challenges in the UCLA study was finding enough participants that fit the definition of Internet “naïve”. But when you average out the curve, with the left being digitally “wired” and the right being “naïve”, there’s no question that the population under the age of 30 (the apparent divide between immigrants and natives) has moved significantly to the left.

Again, let’s come back to my daughters and I. For a 47 year old, I’m remarkably “wired”. Yet my daughter’s use technology in a way I would never think of. I tend to translate my needs into available technological frameworks. I am amazed by that technology. I constantly back “benchmark” against what I remember as a child.  The fact that my email goes instantly around the world still bewilders me. But for my daughters, the technology isn’t amazing. It simply is. They don’t translate, they just use. Of course you use Google to do your homework. Why wouldn’t you? Of course there are a thousand websites specifically devoted to whatever fleeting interest happens to occupy your mind at this point in time. Why wouldn’t there be? Of course I can see exactly what my friends are doing on FaceBook, whether they live next door or in Korea. What’s so remarkable about that?

Stand up comedian Louis CK said, “we live in an amazing, amazing world and it’s wasted on the crappiest generation of spoiled idiots”. Louis is clearly in the digital immigrant camp, and I’m sure you also have to be over 30 to appreciate Louis’s view of things.

The give and take of digital fluency

There is no end to the profound implications that this digital divide could bring to our world. Remember, for everything we get better at through exposure, there are also lost abilities through lack of use. If we become better are navigating online because we spend 10 hours a day on a computer, but we loose the ability to communicate face to face because those 10 hours are spent by ourselves in a virtual cocoon, what are the impacts on our world? Dr. Moody and I touched on this in our interview (full transcript on my blog):

Dr. Moody: I can only comment on this just from personal experience with my children. I haven’t done research on how children interact with the internet. I’ve read some of the papers but I’ve not done any research on that. But it does seem that, you know, they interact more readily and more fluidly. It’s amazing how quickly your kids can navigate across something on the internet compared to how I do. Of course, I’m pretty computer-savvy, I use the computer hours a day. So I think there is a difference between young people and old people. Young people, I think they’ve grown up with it, they accept, you know, MP3 players, cell phones, visual impact touch screens – all that is so natural to them and some of us are still trying to figure out how to program our DVD players.

Gord: Right. But I guess there’s speculation too that as they become more comfortable with technology and it becomes more of a natural extension of how they communicate, there’s potentially a trade-off there. I mean, the whole concept of pruning is that you get better at what you do all the time and you gradually lose capabilities in the things you don’t do very often. And so might this mean, for instance, that the young are losing the ability for face-to-face communication or more kind of focused reasoning over a longer period of time.

Dr Moody: You know, I think that’s a very real concern, and I know that people are looking at some of those issues, attention in particular. The studies that I’ve actually looked at have used computer gaming to enhance visual attention. So we know that you can actually enhance attention using internet gaming practice. But it might be, as you say, that you also have a negative impact for longer periods of attention, like being able to read an entire article versus clicking around and having this immediate visual gratification of changing very quickly. So I’m not aware of the studies that have looked at the negative impact on attention. I’ve actually been looking more on the positive end of how attention has been enhanced and how people are developing computer packages to help children with ADD for instance be able to focus for longer periods of time. But certainly, just it seems that young people have shorter attention spans. I’m not aware of the research, however.

An attention deficit world

As Dr. Moody said, an area of concern is in the fragmented and fast paced online world, the notion of attention is significantly different than it was for us as children. Everything now is bite sized, YouTube length and hyperlinked. Are our children losing the ability to pay attention to anything for longer than 30 seconds? Is the world becoming attention deficit?  There does seem to be proof of that happening. A number of studies have linked excessive TV watching under the age of 3 to increased incidence of ADHD. Remember, this is the critical period of pruning, where the brain is being optimized for its environment. If that environment is one of jump cuts, bright colors, loud noises and a different stimulus every second, the brain will change and adapt to that. Unfortunately, that environment isn’t the real world.

What’s equally ominous is what happens in the second period of extensive pruning and brain plasticity: adolescence. It’s during this time that the brain develops its ability to create social connections and empathize with others. During this process, friends become the most important thing in the world. But what happens when the majority of contact with friends is not face to face, but Facebook to Facebook? The brain has evolved to communicate most efficiently when we’re physically in the presence of our counterpart. Are our children sacrificing their ability to connect?

These are very new areas of research. Academics are just beginning to explore the impact of technology on our culture and society. Ironically, those doing the exploring are digital immigrants. Do the natives care as much as we do? Are we just clinging to things because those are the things we know? As digital natives eventually inherit the positions of power, how might that change things…how might that change everything?

Some time ago, I wrote an article called “Are Our Brains Becoming Googlized?” It became my most read Search Engine Land post ever. Apparently I wasn’t the only one fascinated by the prospect of wholesale rewiring of our brains through exposure to technology.

UCLA’s Semel Institute for Neuroscience and Human Behavior is one of the hotbeds of this brain research, with Drs Gary Small, Susan Bookheimer and Teena Moody doing a number of interesting fMRI studies looking at the impact of technology on our neural networks. One study in particular was fascinating to me, looking at how internet searching activated different parts of the brain. I had a chance to connect with Dr. Moody and ask her more about the study. In today’s column, I’ll share some excerpts from that interview.

The study was conducted with older participants and the goal was to see if the Internet could be used as a way to “exercise” the brain, slowing mental decline. One of the fascinating outcomes was not just which parts of the brain “fired” when searching, but the difference in the level of mental activity between practiced searchers (called the Internet savvy) and newbies (called the Internet naïve). This touched on a number of areas that overlapped with my thoughts and research findings in the past few years.  The interview touched on a number of areas, including some of the methodological challenges of fMRI research. For those of you interested, the full transcript is on my blog.

In this column, we’ll explore possible reasons why more of the brain fires as we become more comfortable with searching. Dr. Moody and I explored some possible explanations for this. Danny Sullivan and I have been telling anyone who would listen that Googling is a habit. This study seems to provide more evidence for that view. But more than this, it’s a fascinating glimpse into how our brains evaluate what we see on the search page.

First, I’ll let Dr. Moody explain the original motivation behind the study, providing mental exercise for the elderly:

Dr. Moody: We have a situation where almost everyone has access to a computer, it can make this almost universal. Especially as we age, we’re not getting out there as much to walk around and some people don’t have the ability to go to senior centers and interact with other people, but that you could do something in your own home without requiring great mobility is very exciting. Also, there would be so much choice, there’s so much variety on the internet, it can be individually tailored to your personal preferences. So in this study I tried to pick topics that might be interesting to older adults – you know, walking for exercise, Tai Chi, health aspects of eating different types of food. I think that if it’s enjoyable for someone and if you don’t consider it to be a job to get out there and stimulate your brain, that people will do it more frequently. So that’s part of what’s exciting about it, is that it should be easily accessible to people once they know how to turn on the computer and activate the internet.

The main objective of the study was to see the difference in brain activity with two different groups between reading text and searching the web. Specifically, the team wanted to see which parts of the brain “lit up” when conducting the tasks. I asked what was the reasoning behind using reading as the comparative task.

Dr. Moody: Well, actually, both for the reading and for the internet and Google searching, we used a different baseline. We had a button-pressing baseline where white bars appeared on the screen and they just pressed the button when a white bar appeared for the location on the screen. And we compared the pattern of activity when they were reading and making… selecting different chapters or when they were selecting Google, from the Google search screen and reading off the internet to that pattern of activity. So our control was more of a low-level control baseline.

Then, in a higher-level analysis, we compared the pattern of activity while they were reading to the pattern of activity while they were doing the internet search. So both tasks had a lower-level baseline control.

The researchers also wanted to see the difference between the Internet Savvy and the Internet Naïve. Some results were as expected, and some came as a surprise:

Dr. Moody: Well, we found that the pattern of activity was almost identical, and that really frankly surprised me at first because I thought that the internet even for the naive participants would require additional areas, because when you’re searching the internet you are engaging in decision-making, you have to suppress extraneous information, so there’s inhibition required. So I was surprised to find that it looks like in both the internet task and the reading task the subjects are just engaging their language areas, their visual areas, there’s some sensory integration areas as well, but it looks like they’re reading in both cases. And not surprising at all about the areas recruited, because they’re language areas, memory areas, and visual attention areas.

But The researchers found something different when they were looking at the internet-savvy group.

Dr Moody:  For the internet-savvy group, their reading areas were virtually identical to the reading areas that were activated for the internet-naive participants, but the very interesting part was the savvy group did recruit additional areas and these were frontal areas that had to do with decision-making, cingulate areas that have to do with conflict resolution. It’s not surprising, it’s what we expected, that these additional areas for decision-making would be required and higher-level cognitive function would be required, and that’s what we found in the internet-savvy group.

So, as you become more comfortable with the Internet, you actually use more of the brain. This is counter-intuitive. Generally as we learn to do something, the level of cognitive effort decreases rather than increases. Think about how hard you had to concentrate when you learned to drive, something that seems second nature to you now. So, in this study, it appeared there was something more happening upstairs as we learned our way around Google:

Gord: To explore that a little bit, we’re seeing that people are actually cognitively engaging with the results – they have to make decisions, they’re comparing them. What happens there? With the internet-naive, obviously they weren’t engaging with the content nearly at the same level, but the internet-savvy… Is there a certain level of fluency with search where you elevate it to a higher level and you’re using that input to make decisions?

Dr Moody:  Yes, that is certainly one interpretation, and one interpretation that we have for the data – that it does require additional areas and as you practice it, you do become more fluent and more expert at it.

Now there are two different schools of thought on this. One is that when you first learn a task, you require greater activity and more attention, and that one could expect higher levels of activity if you were new at something. People with expertise can actually show decreases in their functional MRI pattern of activity. But what it seems here is that while engaging in internet searching, you are still very actively engaging these decision-making areas and it might be that the naive people are overwhelmed by the situation and are just treating it like a book – you’re still not trying to integrate the information, they’re reading it as though they were reading a book.

There’s one other interpretation as well, and that is that internet-naive people just have a different pattern of wiring in their brains from those who are internet-savvy – people who prefer using the internet and enjoy that mode of reading are wired differently from the internet-naive people. And we can’t distinguish that in this study, but that is also a possibility.

Two fascinating intepretations. One, as we become fluent with the search page layout and the actual act of searching, we let other parts of the brain kick in. We start interacting with the results. Two, Google is in fact rewiring our brains. The first is my Google Habit theory, and we’ll return to this. But the second possibility needs a little explanation. As I said in the first column on this, the phrase “rewiring our brains” sounds ominous indeed. The fact is, our brains are constantly being rewired. Rewiring is the basis of learning and memory. But there’s a more fundamental rewiring, building inherent circuitry that goes deeper than simple learning. And it was this that possibility that I asked Dr. Moody about.

Gord: You say they’re wired differently. Would that be the typical, neural “fire together, wire together” wiring that happens when we learn anything, or is this something more fundamental in the pruning that happens during the formative years?

Dr Moody:  Well, certainly in development, you know, we have good evidence that things do wire differently depending upon environmental influences, and definitely there’s evidence now against the old theory that adult’s brains don’t change, but definitely after brain injury there’s been evidence of re-wiring or re-mapping brain regions to overcome deficits. We don’t know what’s happening here. This is a very preliminary study, but one interpretation could be that there was a re-wiring, as people practice on the internet that these areas become more active. But all we can really say is that the pattern of activity is different.

Whatever the cause for the rewiring, it’s significant that as we become more familiar with search, the brain seems to kick in at a different level. It’s almost as if the basics get done on autopilot so we can spend more time considering the results. This aligns with a past blog post I did and it was something I asked Dr. Moody about:

Gord: So one of the things I’ve suspected, when we’ve looked at behaviors in interacting with search, is as you become more used to using search, more comfortable with the interface, you don’t have to worry so much about navigating through the interface, that becomes more like a conditioned, habitual behavior. Which means your prefrontal cortex is free to kick in to do those cognitive assessments, to say, “Okay, here’s what Option A offers me versus Option B,” so it’s almost kicking it up to a higher level of processing. Does that seem to make sense? It’s like I said, Google has become a habit and at some point the basal ganglia takes over and runs it as a habit which frees up the prefrontal cortex to do more heavy lifting.

Dr. Moody:  Well, our data are definitely consistent with that interpretation, and I think that that’s what part of our interest is, is how can we enrich our lives as we age, how can we improve our cognitive function or slow cognitive decline? And so yes, that’s an interpretation we would like to have because we would like to say, “Oh, we can do something to make our brains better as we age,” so that’s very exciting and interesting, and it is consistent, however we can’t conclude that. We don’t have any causality here at all.

So, is Google changing our brains? The answer appears to be yes, but at least at one level, that could be a good thing. Searching does exercise the brain in a similar way to other activities that require evaluation of information, decision making and rational thought. It’s easily accessible exercise that could keep our mental muscles limber as we age. The attractive thing about the internet is that it doesn’t seem like exercise if we’re finding information on something that interests us.

But there’s a more ominous side to this as well. What about the generation that’s growing up with Google? Increasingly, more and more neuroscientists and psychologists are expressing concern that the rapid fire stimulation of TV and video games could be turning our youth into socially challenged, multi tasking digital jolt addicts. The question journalist Nicholas Carr posed was “Is Google Making Us Stupid?”

The Digital Native – Digital Immigrant theory has been espoused by many, including Dr. Moody’s co-researcher, Dr. Gary Small. In my next Just Behave column, Dr. Moody and I will chat a bit about the implications of Digitally Hardwired Brains.

Despite the economic meltdown of the past several months, search still seems to be cruising along with barely a hiccup. To be honest, it’s a little surreal. For those of us in the industry, it seems like we’re in a protective bubble while the world falls down around us. To be sure, the impact has been felt, even in our search world. But all things are relative, and relative to every other marketing channel, search seems invincible.

The question is: Why?

What is it about search that makes it weather economic storms so well? Based on what I’ve seen, the only clients that, once having adopted search as a channel, decide to drop it are the ones that are going completely out of business. In the past year, I’ve seen a lot of budget flow into search from other channels, but almost none flow the other way. Fourth quarter numbers from a number of monitoring services indicated that search volumes and revenue seemed higher than ever. True, the astronomical growth rates of the past have tamed down dramatically, but search is still growing. Name me one other channel where that is true.

So, with apologies to Stephen Covey, I’ve identified 7 habits unique to search that lead it to being seemingly recession proof. Not surprisingly, they all tie back to user behavior.

Habit 1: Search captures demand

Search is unique in its ability to harvest consumer demand. It’s the most efficient mechanism I’m aware of to do so. More and more, all other marketing activities lead prospects to search, where smart marketers can cast a very effective net to capture the potential business. TV ads lead to search. Newspaper and magazine ads lead to search. Direct mail leads to search. Yes, potential business can still find other paths to purchase, but every day, more and more traffic intent on buying crosses through the search intersection.

Now, with this there is a lag effect that marketers have to be aware of. Other marketing creates demand. Search captures demand. The two can’t work independently, at least not in a sustainable way. As overall demand decreases, marketing budgets tend to shift to channels that are more measurable and effective in capturing real revenue. Search benefits from this in the short term, but in the long term neglecting the creation of demand will have an inevitable impact on the amount of demand that can be captured on the back end. In a sustained downturn, all boats will sink eventually, just not at the same time.  But, for six other reasons still to follow, I believe search’s vulnerability is far less than other channels.

Habit 2: Search is category agnostic

As consumerism fails, so too does advertising. At least, that’s the theory. But humans have two polar orientation points that guide our buying behaviors. We alternate between the expected rewards of gain and prevention of loss. Both can translate into consumer activity, although the former generates far more activity than the later.

The best way to explore this is with an example. If the economy goes into free fall, advertising luxury vehicles is probably going to be an exercise in futility. However, debt consolidation might find a very receptive market. Search can cover both categories quite effectively. Search volumes will naturally switch from promotion type queries (looking for things to buy) to prevention type queries (looking to hold on to what you have) and queries equal inventory. In an economic downturn, search is rather unique in that another type of inventory for another market is automatically created. And it’s an inventory that will be attractive to a select group of marketers.

Habit 3: Search is democratic

Search campaigns are relatively easy to launch. A search campaign can be up and running in a few minutes. Also, the pricing model of search allows almost anyone to play. So, for any given keyword, you can find advertisers running the gamut from huge, multinational brands to the mom and pop shop down the street from you. This opens a huge pool of potential advertisers that can place their ad budgets on search. Search engines have the luxury of spreading their risk over a much larger pool of customers. If GM cancels their Super Bowl ads, there are a limited number of candidates that can pony up the budget required to grab those 30 second spots. But with the efficiency of the search marketplace, if GM turns down their search spend, other players automatically rise up to take their place. Yes, click prices may trend slightly downwards, but at least it’s not a gaping black hole that can’t be filled.

Habit 4: Search is proactive and interactive

The previous 3 habits had more to do with the nature of advertisers than users, so let’s explore why search is so effective in connecting with prospective customers. Search demand stays strong because it’s demand expressed by the user. If a marketer spends money on a TV ad, they have no idea if anyone is listening, or, if they are, do they care? It’s a gamble based on a rather cloudy mix of rating points, consumer confidence indexes, market research and a number of other fuzzy variables.  Compared to that, search volumes are hard and crystal clear numbers: real people expressing intent by searching for things.  Short of a customer giving you their credit card and telling you to charge away, there are few more concrete signals of consumer intent.  The path from a search click to a customer conversion can still be tricky, but it’s an arrow straight and short path compared to the one that starts from watching a TV ad or reading a magazine.

Habit 5: Search gets our attention

We consciously decide to search. We have a goal that leads us to our favorite search engine and our brain is engaged in the pursuit of relevant information. That means advertisers don’t have to get clever to get our attention—it comes by default with the act of searching. All advertisers have to do is three things:  1) catch our eye on the search page, 2) deliver a message that convinces us to click, and 3) deliver a relevant and compelling path to follow when we get to the landing page. If you’re deciding where to place your advertising dollars to get the best return, the head start that Search has by coming out of the gate with our attention already captured is a huge advantage.

Habit 6: Search is ubiquitous yet focused

Almost everyone I know uses search. In fact, almost everyone I know uses Google. And I can’t think of one other ad channel I can say that about. A few people I know watch The Office or 30 Rock. A few people I know read the newspaper or Time magazine. And increasingly, even fewer people I know listen to the radio.

Everybody searches. And, in the future, they’ll just search more. Right now, the majority of those searches are funneled through Google. Name one other channel where you can reach almost everyone, and you can reach them at the precise time they’re thinking of your product. That’s one extremely focused intersection that almost your entire market may walk through. Other channels can only dream of this combination of reach and targeting.

Habit 7: Search is friction-free

Finally, search is easy. It’s friction-free. You don’t have to worry about production costs or media buying formulas. New advertisers can adopt search without stumbling over a huge learning curve or sunk set up costs. There are virtually no barriers to entry. The biggest hurdle to more advertisers using search is simply lack of awareness and education. And, ironically, the current economic situation will help Search in spreading the word. In the current economic environment, stupid advertisers won’t survive. They will have to become smarter and more efficient. They will have to evolve faster than the competition. And that means they’ll be finding more effective ways to advertise.

There you have it. 7 reasons why I believe Search will do all right, even in the worst economic crisis of our lives. Will there be bumps and scratches? Yes, but no fatal body blows. Unfortunately, the same won’t be true of other ad channels. Having started my career as a radio copywriter, which then led to other traditional marketing channels, I have to say I’m counting my blessings that internet search came along and looked like an interesting area to pursue. In fact, the reasons I made the switch in 1996 were the same ones I shared today. I believed then that search was a fundamentally important shift in the practice of marketing, and I still believe so now.

In new numbers just coming out from AdGooroo, it looks like Microsoft is closing the gap on Yahoo in terms of advertiser share for the last quarter of the year, moving to a narrow three point difference (19.4% vs 16.4%) from a 17.6% gap in the third quarter. Also, in the report, it seems that Google was fairly aggressive in cranking up ad coverage while Yahoo stayed flat and Microsoft actually backed off on ad coverage, focusing on ad quality control and a more holistic search experience.  Finally, AdGooroo is forecasting the strongest quarter ever for Google and Microsoft, while showing Yahoo as fairly anemic, probably due to advertiser’s doubts about Yahoo’s future.

So, what does this have to do with user behavior? Well, it’s simple. As go the ads, so goes the user experience. And these shifts in the marketplace could be very indicative of correlated shifts in user experience. It seems like Microsoft is finally getting the message while, ironically, Google seems to be willing to experiment with one of the core tenets of their success.

The Motivation behind Microhoo

Of course, we’re freshly abuzz with Microhoo rumors and the AdGooroo report indicates that timing couldn’t be better.  They also pulled out a very telling Ballmer quote from December 5^th:

“We’re fully prepared to compete without any partnership with Yahoo. We don’t need to act. Would it be advantageous for both of us to make a deal? Look, the fundamental basis for doing the search deal with Yahoo has to do with critical mass in the advertising marketplace. It doesn’t have to do with technology, or any of these other things, it really is a market phenomenon. Together we would have more advertisers… which means we’d have more relevant ads on our page. We’d have higher monetization levels possible in front of us because there would be more people bidding on more key words. Most importantly, Google would have perhaps a real credible competitor sooner.”

Ballmer is talking about advertiser critical mass and Yahoo’s ability to contribute significantly to that critical mass seems to be evaporating quickly, if the AdGooroo numbers are any indication. If this is the sole motivation for Microsoft to do a deal, Yahoo’s marketability has a very narrow window indeed. I won’t talk about the value of Yahoo’s other assets or any of the nuances of a potential deal. That’s not the purpose of this column. But there’s another very interesting line in here, from the user perspective, and to understand the importance of it, I have to share the results of an astounding study we did at Enquiro for, you guessed it, Microsoft.

First, let’s read between the lines in Ballmer’s quote:

Together we would have more advertisers… which means we’d have more relevant ads on our page.

Ballmer goes on to talk about monetization, but what about perceived relevancy on the part of the user?  Some time ago in this column I talked about the importance of relevant ads, especially when they’re at the top of the Golden Triangle.  If poor ads showed at the top, we knew anecdotally that it impacted the user experience. We found empirical evidence of this in our second eye tracking study, when, in the midst of the study, Microsoft was switching from their Yahoo ad partnership to showing ads from their own inventory. The MSN (this was pre-Live) inventory was a fraction of Yahoo’s, which meant the relevancy of top of page ads took a major hit, right in the middle of our data collection.

When we started analyzing the data, we saw a significant change in user behavior over the MSN sessions. At first, we didn’t know what was going on. In a number of cases we saw virtually no interaction with top sponsored ads. It was almost as if banner blindness was occurring. We saw 86.7% of the sessions had their first fixation in these top sponsored ads, but only 55% stuck around to start scanning. This was a significantly higher drop off rate than we saw on Google or Yahoo.

Enquiro Eyetracking II Study, 2006

When we realized that in the affected sessions, the ads were coming from a different inventory, we divided the sessions into two groups for further analysis. When the ads came from Yahoo’s inventory, users spend an average of 4.93 seconds looking at them, which represented 41.6% of the total time on the page. These ads also captured 42.86% of the click throughs. But when the ads came from Microsoft’s inventory, users spent an average of 1.5 seconds on them, representing just 11.48% of the total time on the page. And these ads captured only 5.8% of the click throughs.

So, obviously poor ads meant much lower monetization on the search page, which is what Ballmer referred to. But this just meant users were skipping over the ads. How much of a negative impact did that have on the overall user experience? That was the question Microsoft asked us to prove in a subsequent study.

The study design was simple. We created a realistic search experience, then showed a mocked up results page and asked participants to interact with it as they would normally. Afterwards, we asked them four questions:

• Would they use the search engine to launch a similar query?
• Would you use the search engine for other queries?
• Would you recommend the search engine to a friend?
• Would you make the search engine your preferred search engine?

In the study, we divided the participants into two groups that were shown slight different versions of the search page. The only difference was the ad that showed in the top sponsored location. In the first group, the ad was a high quality, relevant ad. In the second group, the ad was less relevant (although not completely non-relevant) and of lower quality. That was the only difference. The other 16 results, including the other ads and algorithmic results, were exactly the same on the two versions. As I said, the results were amazing.

Enquiro & Microsoft, 2007

Just one ad, in a critical position, could substantially impact the user’s perception of the quality of the search experience. As I’ve said, over and over, in search, perceived relevancy is much more important than real relevancy, and with the current layout of the search page, a huge factor in perceived relevancy is whether the first thing you see, which is usually an ad, is a relevant match to your query.

If you take the results of both of these studies, it shows how important a big ad inventory can be for the overall user experience. The more ads you have to choose from, the more likely you can show a highly relevant one in that critical Golden Triangle real estate.

Microsoft’s on the right track

Finally, it appears the message is getting through at Microsoft. If the recent AdGooroo numbers are any indication, it appears that the importance of top of page relevance is being understood in Redmond. And, once it is, it creates a virtuous cycle that simultaneously improves user experience and monetization. This is something Google has always understood, but those same AdGooroo numbers point out a troubling trend in Mountain View which I hope is just a product of seasonality in advertising.

AdGooroo 2009

If top of the page relevancy is so vitally important, a search engine has to abide by this rule: If you don’t have a highly relevant ad, don’t show it at the top of the page. In the past, Google has always had a much firmer grasp on this rule than Microsoft or Yahoo.  But if we look at trends in ad coverage across the three from AdGooroo, it looks like the grasp may be tightening in Redmond (Microsoft) and slipping a bit in Mountain View (Google). Microsoft looks to be actively tightening up ad quality, resulting in fewer ads being shown, while Google seems to be experimenting with showing more ads. Yahoo continues to be the most aggressive in showing ads.

What isn’t shown in the above graph is how many of these ads show up in the critical top positions. I can say anecdotally that Google seemed to get more aggressive in this as well in the last quarter, while Microsoft seemed to be backing off. There are a number of complex and interrelated user behavior factors at play here that are beyond the bounds of this column, but showing ads at the top of the results is probably the single biggest critical factor in perceived relevance and, therefore, user experience. Someday, Ask and Yahoo may realize this, but I suspect it will be too late. I doubt they’ll ever make the correlation between this and their declining respective market shares.

As I’ve said, it’s not really surprising that Google’s ad coverage increased during the fourth quarter, given the retail importance of the quarter. It will be interesting to see if the increased aggressiveness persists through 2009.

Back to Ballmer’s critical mass

To sum up, relevant ads, shown appropriately, are a critical user experience factor. And the bigger the inventory, the greater your ability to consistently deliver relevant ads. If you take Ballmer at his word, that’s the only reason Microsoft wants Yahoo. Let’s again use the AdGooroo numbers to see what this window of opportunity looks like.

AdGooroo 2009

It appears that Yahoo is trying to sell cruise packages on the Titanic. Obviously, both Google and Microsoft are picking up the advertisers that are jumping overboard.  The question is, can Yahoo bail out before it sinks forever?

For more on the latest AdGooroo numbers, go to their press page.