When looking to predict how MBA students and analysts would find information in a digital environment, Peter Pirolli found his answer in an unlikely place: animal’s foraging patterns. Pirolli, working at the Palo Alto Research Center, was trying to predict with some mathematical accuracy the behavior of humans when searching for information but was having challenges finding models that accommodated “messy” data in an unstructured environment. Given the fuzzy nature of the problem, it seemed that there wasn’t an overlying rational to human behavior. It wasn’t until he applied the principles of Stephens and Krebs optimal foraging theory (1986) in animals that the patterns started to fall into place. Pirolli refered to it as his “ah- ha” moment.
Food for thought
When Pirolli first published his findings (1992) not all shared his “ah-ha” moment. There were more than a few detractors. After all, in the tradition of others trying to accommodate human behavior in the existing rigid and rational intellectual frameworks (Tversky and Kahneman, Simon, etc) Pirolli was breaking new ground here, drawing from several academic areas to try to explain human behavior. But, as academics began to open up to the idea that humans don’t always act in rational ways and every human behavior is built on inherent structures that were adapted for an environment that bears little resemblance to our present one, acceptance of Pirolli’s theory began to spread. Later, together with Stuart Card, Pirolli published the often cited paper that laid the foundations for Information Foraging Theory.
Pirolli and Card’s information foraging model makes perfect sense from an evolutionary perspective. It’s not like we came equipped with hardware for navigating online information environments. So we had no choice but to adapt existing capabilities to new challenges. For reasons I’ll discuss next, there are more than a few parallels between looking for information on the web and looking for food on the African savannah.
The fact that it took us this long to realize we seek out information the same way we look for food is due to two factors: first, the web presented us with an information structure unlike anything we had encountered previously and secondly, a large percentage of our activities in both cases, looking for food and looking for information, is driven by subconscious behaviors rather than rational ones at the cortical layer. Like so much of what we do, the way we seek information is driven by heuristic rules buried in our subconscious minds.
The messy, organic web
The web presented an interesting challenge to humans looking for information. Previous to the web, our data was stored in structured, hierarchal, cataloged depositories. Access to the information followed logical, rational patterns, whether it was alphabetical sorting (encyclopedias) or the Dewey decimal system (libraries). Information was gathered and sorted, overseen by rational, cortical thought. This meant access to that information could follow those same logical, rational paths.
But with the web, information became organic and messy. It cropped up here and there, without any overseeing archivist to make order of it. Information on the web became more like the real world, an emergent, imperfect, living environment. The information we sought could (sometimes) found in clusters of varying quality spread across the web. Information was “patchy”, inconsistent and there were no guarantees as to the quality. There was much to be found, but we were on our own and we entered at our own risk. This meant new strategies were required to seek out the information we needed. Or, as Pirolli discovered, it meant we resorted to much older, inherent strategies. We dug deep into our human hardware to find skills and gut instincts that could be adapted (or, in evolutionary terms, exapted) to this new challenge. We resorted to foraging.
In optimal foraging theory, the foundation of information foraging, the simple rule is: don’t expend more energy finding the food than the food provides. We have evolved some very sophisticated heuristic mechanisms to ensure we don’t invest too much energy in pursuing unpromising sources of food. These same mechanisms seem to be at play when we look at information.
First of all, the diet is important. Is the information we’re looking for scarce or prevalent? How much information are we looking for? And what is the nature of that information? We begin foraging with a existing idea of the type of information we’re hunting and some idea of its likely location and availability. This helps shape our search strategies and our level of patience with any information “patches” we encounter. If we believe the information we seek is prevalent and easily attainable, we’ll have little patience with unpromising patches. But if we know it will be scarce, we’ll dig deeper to find it.
We seek patches of information online by following information scent. Just like in biological foraging, scent provides us with clues to the optimal path to follow to our prey. Remember, the shorter the path, the better. We are programmed to expend as little energy as possible in the pursuit. So in evaluating scent, we evaluate the strength of the scent (determined by relevancy), the promise of “patch” richness (how much relevant information are we likely to find) and the distance of the “patch” (how difficult is it to get there).
But how do we know when it’s time to give up on a patch and start looking for a more promising one? In biology, there’s a risk/reward equation called the Marginal Value Theorem, first put forward by Eric Charnov in 1976. As you start consuming food in a patch, its future value decreases with each bite. You have to balance this against the energy expenditure required to find another patch and the risk that comes with not knowing how rich the next patch is or how far you have to go to get there. These complicated calculations and assessments are done at a subconscious level, with us making gut decisions about when it’s time to move on.
Online information foraging differs from food foraging in that there’s no physical energy expenditure required. Distances between websites are measured in clicks, not feet or miles. But time is an equally precious resource, and we are very careful about committing it in our pursuit of information. It is our drive to conserve time that made the introduction of search engines such a dramatic turn of events in information foraging. And that’s where I’ll pick up in the next Just Behave column.
Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.