Scanning Consideration Sets And Their Importance To Search Marketers

Just Behave - A Column From Search Engine Land As humans, we have a limited capacity to consider our alternatives. Within our brains, there is a limited amount of shelf space available for us to temporarily put our options on to examine them and make our decision. We have cognitive limits to the amount of information we can process at any given time. When it comes to memory capacity, it seems that the limit for humans is seven, plus or minus two. This comes from a 1956 paper by cognitive psychologist George A. Miller called The Magical Number Seven, Plus or Minus Two. He referred to it as channel capacity, the number of cognitive and perceptual tasks we can successfully undertake as humans. It seems that we come with a predetermined number of “slots” available for us to temporarily swap in items that we wish to consider.

This means that in any activity that requires us to consider alternatives, there is a natural limit that’s imposed on us.

In Miller’s studies, he noticed that the memory span of young adults was around seven elements, which he called chunks. Later, researchers found that the number can vary depending on which category of information makes up the chunks. We have a little more capacity for digits, a little less for letters, and even less for words.

So, in interacting with the search engine, we have to first start with the knowledge that we don’t have enough memory slots available in our own human RAM to consider all the alternatives presented to us on the page. We have to short-list them into a smaller consideration set.

The rule of three

Secondly, as we visually scan information there is another limit that comes into play. We tend to visually scan things in groups of three or four. In presentations, this has become known as the rule of three. This is a behavior that came clearly to light in our latest eye tracking study, comparing interactions on Yahoo, Google and Microsoft Search.

Now let’s take both those principles and apply them to an interaction with a search results page. Here’s how the two tend to work together. We scan three to four listings at a time, which are temporarily loaded into our memory slots. From that first group of three to four listings we make a determination if any of them are relevant to the query we just launched. About 50% of the time, we make our selection from those first three or four listings and we click on one of them. If we don’t find what were looking for in this first can, then we continue to scan down the page, slicing off our second consideration set of three to four listings, again loading them into our memory slots so we can compare them and make our choice. We will continue this way down the page until we either find what we were looking for or give up and try another search.


Finally, let’s mix in the almost universally seen search activity (at least in North America. We haven’t tested this yet in other cultures) of orienting in the upper left-hand corner and starting our scanning activity from there. So now we know where that all-important consideration set will be. It will be to at the top of the page, in the Golden Triangle results.

The scan for scent

Let’s take a further look at the scanning activity that happens on this initial consideration set. As I said, almost everyone orients themselves in the upper left corner. From there they do a vertical scan down near the left side of the listings to look for information scent in the titles present. Generally the hint of scent we would be looking for is the search query, bolded and near the beginning of the title. If we pick up scent, then we would probably scan horizontally across the rest of the title and pick up some more information scent from the snippet or description that lies below the title. We start at the top and we work our way down in this fashion. This creates the well-known “F” shape scanning pattern that we commonly see which in turn creates the Golden Triangle.


We have also seen strong evidence that most users will be accepting of sponsored ads in these top spots, but will want to include at least one organic result in this initial consideration set, so the user can compare the relevance of the organic alternative as compared to the sponsored alternatives.


This level of acceptance for sponsored advertising varies depending on how commercial the query is that the user has launched. In one of our studies, we saw that for highly commercial queries, while both sponsored ads in the north position and the top one or two organic listings were scanned by almost all participants, the click-through’s were almost evenly split between top sponsored ads and the top one or two organic listings. But when we asked participants to interact with the search engine to do consumer research, while they still scanned the top sponsored and the first one or two organic listings, they all made the determination that the top organic listings were more relevant and these was the links they chose.

So why is this important to search marketers? Well, it gives us some clue as to what tends to happen on that search results page. We can predict with a reasonable degree of certainty that the first scanning activity will happen on the top three or four listings. Again, our research has shown that interaction with those top listings depends somewhat on the engine that you’re searching on. Let’s look at two different examples.

Consideration sets and golden triangles

On Google, top of the page relevance is a sacred cow. Google is very particular about what it shows in this initial consideration set. It has to be highly relevant to the query. If the query is such that top sponsored ads may not offer the most relevant option, top sponsored ads won’t appear. Google’s goal is to ensure that whatever appears in the top four results, the odds of one of them being the most relevant result to the query you just launched has to be extremely high. So when we look at this as it plays out in an eye tracking heat map, we see very clearly this top of paid scanning activity.


Now let’s look at Yahoo. During the period of the eye tracking study, Yahoo was not using any type of quality score for sponsored ads, so what appeared in the top sponsored was purely decided by how much the advertiser wanted to bid. Also, Yahoo was much more aggressive than Google in showing top sponsored ads. They appeared for over 80% of the searches, and while Google would commonly show an average of just over two sponsored ads per query, Yahoo was showing well over three ads on the average. So there was a lot more top of page real estate devoted to sponsored ads on Yahoo then was the case on Google. The result on Yahoo was that we actually saw two consideration sets, rather than the one we saw in Google. Users form their original consideration set but because of Yahoo’s aggressive presentation of sponsored ads, this first consideration set usually only included sponsored ads. This fought the user’s natural inclination to compare the relevancy in the top sponsored ads with the relevance found in at least the top one or two organic listings. For this reason, users were then forced to include a second consideration set that included the top organic listings in Yahoo. Again this pattern can be clearly seen in the aggregate heat map we saw on Yahoo.


Takeaways for search marketers

So what are the implications from a search marketing perspective? Well, first of all, it clearly identifies the nature of user interaction with the prime real estate on the page. Understand that the majority of the interaction will be with the top three or four listings on the search results page, and from these top three to four listings, whether they be sponsored ads or organic listings, over 50% of users will click on one of them. The rest of the page combined will not yield the same number of clicks.

Secondly, we have to understand how users will make this determination. They will do a quick scan, generally in about two to four seconds, and will pick up enough “scent” from the listings provided to make their determination on which of the alternatives offers the greatest promise of delivering what they’re looking for. The nature of this scan will follow the classic “F” shape, starting with the top of the page. So if we hope to catch the user’s attention, we have to have a clear idea of the criteria they’re using to determine information scent. Again, this goes back to our notion of semantic mapping that I’ve talked about numerous times before. And, most importantly, we have to understand that the user is not considering our listing in isolation but as part of the consideration set, so we will be competing directly with the other two or three listings that we share this all-important real estate with.

Finally, let me look at one of the question that seems to keep coming up in various sessions that I speak at. If you’re in the top sponsored location, do you need top organic visibility and vice versa. Again, let’s consider the dynamics of the interaction with this top consideration set. And for the purpose of illustrating it, let’s assume that a company has both the top sponsored location for the query and the top organic. The user has sliced off a consideration set of four listings to consider, some of which are sponsored and some of which are organic. By owning the top spot in each, you are now controlling 50% of real estate that user is going to be interacting with. And you’re reinforcing your message with twice as much information scent as anyone else in the consideration set. This is why we often see the multiplier effect that happens when you do own top spot in both locations. The whole is greater than the sum of its parts. Of course, it opens the door to possible cannibalization of traffic you might be getting any way. Again, there are no universal rules here, it’s completely dependent on intent of user and what else appears in the consideration set but you can begin to see why this duplication of presence can be so powerful in attracting the attention of a user.

When it comes to the search engines, this notion of a consideration set becomes even more important in the user design process. As we gain more understanding about how users interact with search results, it should come as no surprise that all the major engines are paying more attention to the top of page real estate and the initial consideration set. The introduction of quality scoring in Yahoo is as a direct result of a further understanding of the importance of relevancy in the initial consideration set. The higher the degree of relevance and quality found in these ads, the more successful the user experience.

Gord Hotchkiss is CEO of Enquiro, a search marketing firm that produces search engine user eye tracking studies and other research. The Just Behave column appears Fridays at Search Engine Land.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: Strategy | Google: User Interface | Search & Usability | Stats: Search Behavior | Yahoo: User Interface


About The Author: is CEO of Enquiro, a search marketing firm that produces search engine user eye tracking studies and other research.

Connect with the author via: Email


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Bill Slawski

    I hate doing this, but I have to disagree, Gord.

    The rule of seven isn’t meant to apply to the amount of scannable material that we have in front of us as we are working. Even George Miller called that a misapplication of his research. A short snippet from Dr. Miller in his response to being told that his work was being used in a fashion like you are suggesting (in the context of billboards only containing seven items, so that they don’t cause accidents if they have too many more):

    But the point was that 7 was a limit for the discrimination of unidimensional stimuli (pitches, loudnesses, brightnesses, etc.) and also a limit for immediate recall, neither of which has anything to do with a person’s capacity to comprehend printed text.

    Here’s a link to the original paper, rather than the wikipedia entry:

    The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information

    Please note that even the Wikipedia page discusses the urban mythology surrounding Dr. Miller’s research. Armed with this, does it make you come to some slightly different conclusions? I hope so.

  • Bill Slawski

    Just a little more, Gord.

    From George Miller’s paper, the conclusion:

    And finally, what about the magical number seven? What about the seven wonders of the world, the seven seas, the seven deadly sins, the seven daughters of Atlas in the Pleiades, the seven ages of man, the seven levels of hell, the seven primary colors, the seven notes of the musical scale, and the seven days of the week? What about the seven-point rating scale, the seven categories for absolute judgment, the seven objects in the span of attention, and the seven digits in the span of immediate memory? For the present I propose to withhold judgment. Perhaps there is something deep and profound behind all these sevens, something just calling out for us to discover it. But I suspect that it is only a pernicious, Pythagorean coincidence.

    These pages are also worth a look on the subject:

    Edward Tufte’s discussion on: The magical number seven, plus or minus two: Not relevant for design

    And, a link on that page leads to this paper:

    How magical is Miller’s famous Seven for HCI issues? (pdf)

    James Kalbach’s article is also worth a look:

    The Myth of “Seven, Plus or Minus 2″

    Denny C. LeCompte’s paper:

    3.14159, 42, and 7±2:
    Three Numbers That (Should) Have Nothing To Do With User Interface Design

    A snippet from that one:

    At best, Miller’s 7 ± 2 figure applies to immediate serial recall for a sequence of familiar, easy-to-pronounce, unrelated, verbal stimuli presented auditorily with no distracting sounds within earshot. Thus, the narrow range of generality implied by the research findings cannot support the wide variety of situations to which people try to apply this heuristic. Based on the relevant data, user interface designers should probably forego application of the 7 ± 2 heuristic altogether.

  • Gord Hotchkiss

    Bill, you made a good point, but I think there are two distinct things here. One is our ability to recall distinct pieces of information and one is our ability to scan and assimilate visual displays. I refer to them seperately in the column. These are two cognitive limits that play a part in our interactions with the search engine, and they both impact the nature of that interaction. I’m not trying to imply they’re interchangeable.

  • Bill Slawski

    Hi Gord,

    The cognitive limitations that Miller discusses are within a completely different and much more limited context than the one that you present, and yet it seems like the basis for some of your research, which is why I couldn’t help myself but bring it up.

    When you write, “Now let’s take both those principles and apply them to an interaction with a search results page. Here’s how the two tend to work together,” it does seem like you are attempting to use Miller’s research to stand for a limitation of a person’s cognitive ability to scan and interpret materials directly in front of them. It’s the same context as the concern about billboards that caused Miller to state that his research was being misinterpreted.

    Isn’t it just as easy to hypothesize that a searcher expects to see relevant results within the top few links to pages that appear after a search, and that after looking at the first three or four may decide that the results do or do not hold value, and may then decide whether or not to revise their query? Thus, not being a matter of limitations of our ability to scan, and to retain a certain amount of information within our human memory banks, but rather to be able to review and discriminate about the relevance of results based upon a review of the first few results?

    Doesn’t the fact that we read left to right and top to bottom also play a role in the shape of those results? That we go from top to bottom when we read would also play a role in that shape. If the first three or four results don’t look promising, there seems like little value in proceeding too much further down the page, regardless of whether my ability to scan and recall is tied to some magical number or not.

    So, do we only scan the top few results because that’s all we are capable of, or do we scan them because we have the ability to make a judgment based upon information directly in front of us, that we don’t have to necessarily recall after the fact? In other words, what you are calling a limit, I’m calling a decision.

  • Gord Hotchkiss

    Good point. I think the fact that most of us find something relevant enough in that top consideration set to at least click on is highly pertinent to the interaction. Thanks for adding your insight!

  • Bill Slawski

    Thank you, Gord.

    I’m really happy to see someone exploring search from the perspective that you are. I may question some of what you are doing a little, but I really appreciate the level of inquiry that you are bringing to the topic of search.

  • Andy Edmonds

    Miller’s work was focused on the auditory loop and memory for acoustic data.

    There is some critical importance to the 7+-2 quantity that seems to pervade the cognitive system, but for the phenomena described here, you’re much better off studying the “subitizing” research that specifically looks at visual attention:

    It’s amazing how even professionals trained to PhD levels in cognitive psychology promote an over-interpretation of Miller’s (ancient) work. See Just and Carpenter’s theories on working memory for a more recent take in which both storage and processing are considered a single resource.


Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide