Last week on Just Behave, I talked about the importance of consideration sets in search; how we tend to slice off three or four top listings in our consideration set at the beginning of our interaction with the search results page. Today I want to talk about another concept, no less important, that looks at the very first second of interaction with the search results page. I’ve referred to it in the past as the Area of Greatest Promise.
Start High and To the Left, Work down from There
As I mentioned last week, there is an almost universal behavior amongst North Americans when they interact with page of search results. The very first thing they do is move their eyes from wherever they entered the page up to the upper left corner, usually orienting ourselves on the border that tends to delineate the search results from the logoed area above. We use this as our anchor point and from there we generally start our scan path through the results page. What this means is that this is almost always the area we start looking at to pick up information scent. It’s in this very small triangle (shown below on a typical heat map showing the first half second of interaction with the search results page) that we begin picking up the cues of relevance on the page.
The Golden Triangle extends beyond this very small area but this is very much the initial interaction from which the Golden triangle is defined.
What I’ll do today is look at how five different search engines treat this very valuable area of greatest promise and why quality scoring in top sponsored ads should hopefully contribute to a better user experience on the engines that implement it.
The Practically Perfect Search Engine
First of all, let’s talk a little bit about the ideal user experience on a search engine. If we somehow created the perfect search engine, it would always provide us with exactly what we’re looking for right on top of the page of results. And we could determine that it was relevant to our query by a quick glance at the beginning of the listing. As a society, we are conditioned to scan written material in a left to right, top to bottom way. Therefore, it’s become ingrained that we start scanning in the upper left corner. Further conditioning on the part of the search engine has taught us that we should always expect the greatest relevance to our query at the top of the page. So this very small triangle is where we expect to find what we’re looking for. If we don’t find what we’re looking for here, we extend our scanning to begin looking for relevant cues on the rest of the page.
As I said last week, the scanning tends to continue first down vertically along the left-hand side and then across, horizontally, looking for information scent. But my suspicion is that we form a very quick opinion about the relevance of the search results based on what we pick up in this very narrowly defined area of greatest promise. Even in our first split second interaction with the page, where we’re all interacting with this tiny piece of real estate, we start to form opinions about whether this was a good set of search results or not. Therefore, in an ideal world, we would always find exactly what we were looking for in this area. We would never have to look any further, so a heat map of this might look like the one pictured below.
The 60 Minute Research Project
If we accept the fact that this is universal behavior, let’s take a look at how each of the five major engines delivers on this promise to their users. Are they providing something approximating an ideal user experience on their search engine? To help drive the point home we did a quick, impromptu research project in our office this morning. We took a query; in this case "Digital camera reviews, and asked a few people what their intent would be if they used this query.
The answer, not surprisingly, was that they would be looking for objective information about digital cameras to help in a future purchase (either online or offline). We also asked if they would be open to clicking on sponsored listings that appear to offer what they were looking for. Almost our entire informal sample said that they would be open to it, but they would probably also click on an organic listing as well. Over and above the actual query, "digital camera reviews", they would be looking for words like unbiased, objective, user reviews, professional reviews, comparisons, and some indication that a large number of cameras had been reviewed. The ability to compare prices was also stated as something that might get their attention. This roughly approximates the semantic map that may be in place surrounding this particular query. Of course, it’s always difficult to disambiguate what every user’s intent might be given any particular query, but in this case we feel reasonably comfortable that what we got from our ad hoc sample would probably represent the intent of the majority of users.
Next we took the results that tended to appear in Golden Triangle real estate across all five engines in both sponsored and organic positions, stripped out formatting and position and asked our users to indicate which of them might appear to offer what they’re looking for. These were the winners, in order:
- Digital Camera Reviews – Unbiased pro and owner reviews plus 100s of merchant quotes on cameras
- Digital camera reviews – Find out which Digital Cameras are best with our unbiased reviews.
- Digital cameras, digital camera reviews, photography views and … International website for digital photography. Up-to-date specifications of digital cameras, latest news, online tutorials, digital camera reviews
- Digital Camera Reviews and News: Digital Photography Review … -Current digital photography news, digital camera reviews, articles and discussion forums.
You’ll notice "unbiased reviews" was a common element in the top two choices.
Finally, we asked them to indicate where they would expect to find the most relevant result. 80% said in the top organic results.
Okay. It wasn’t randomly selected sample, the sample is undoubtedly biased, seeing as it’s made up of employees at a search marketing company and the methodology leaves a lot to be desired, but it’s the best I could do given the hour I had this morning when the idea first popped into my head. This informal study did, however, past the gut check test. We got the results we were expecting to see. So we can start to create a picture of what the desired intent of an average user, searching for "Digital camera reviews", might be. We begin to understand where they might look and what might provide relevant clues as they’re scanning. Now, let’s look at how each of the top five engines delivered on that intent in the area of greatest promise. We assigned an unofficial rating based on the following factors:
- Number of matches with semantic map within the area of greatest promise.
- Bolding of actual query within the area of greatest promise
- Appearance of organic results within the area of greatest promise
And the Survey Says….
Let’s start in alphabetical order and see how our engines did:
AOL – 3 out of 5 stars
When we look at the area of greatest promise, defined by the orange triangle, we see that on AOL it falls entirely within the top sponsored ads. The second of these ads actually offers great scent match to our hypothetical user’s intent and turned out to be the winner of all the listings presented in our informal survey. But this is more a matter of a savvy marketer keyed in to the semantic map for their target market than AOL offering a superior user experience. That said, while AOL didn’t include an organic listing in this area greatest promise, it wasn’t too far removed (shown by the orange box). For this reason AOL received a passing score, although that’s more a matter of the quality of the ads based on relevance to intent then the layout of the search results page.
Ask.com – 2 out of 5 stars
In Ask’s results page configuration, they actually move their vertical results to the top, above their sponsored ads. If I was searching for a particular type of camera this would probably be a good reinforcement of relevancy. But in this particular case, with the query "digital camera reviews", I think it’s a little off base. It’s not what the average search user would be expecting. There was no bolding of my query anywhere to be found in the area of greatest promise and the majority of the prime real estate was used up for the graphic thumbnail of one particular type of camera, again, probably not what I would be expecting to find. The result that I would be looking to find, the top organic result, is a substantially down on the page.
Google – 4.5 out of 5 stars
Google was the only one of the engines that did not present top sponsored ads for this particular query (at least at the time and location of my search). For this reason it received top marks for the inclusion of organic in the area of greatest promise. It also received high marks for bolding the query used and having it right in the upper left corner. It also had matched with the hypothetical semantic map, although ironically, this organic listing did not have as many matches as some of the sponsored ads we saw on the other engines. All in all though, it appears that Google was the winner when it came to delivering in the area of greatest promise, based on the intent of the user.
Live Search – 3 out of 5 stars
Microsoft Live Search also moved a lot into the area greatest promise, at the expense of the organic listings. Once again, Microsoft had the advantage of having one ad appear, in this case from Consumer Reports.org, that was a great match with user intent and probably would’ve had a high click through rate. Like Ask though, Microsoft decided to move their vertical results up the page and in this case, given the query used, they may not be the best match to user intent. In this particular search, Microsoft’s delivered one ad in the area of greatest promise that was a good match with intent, but it happened more by luck then by the design of Microsoft’s usability team.
Yahoo – 3.5 out of 5 stars
Yahoo has come a long way in delivering promise at the top of the page. They’ve cut down on their aggressive presentation of top sponsored ads and were actually able to move the top organic listing to the very bottom of the area of greatest promise. A couple of quibbles with the presentation on the page though. I’m not sure such a large portion of this prime real estate should be used to provide alternative search suggestions. To me, they should be more of a post-scan suggestion that a pre-scan one. I prefer Google’s choice to show these search query alternatives at the bottom of the page rather than the top. The other quibble has nothing to do with Yahoo’s page layout, other than policing the quality of the top sponsored ads. In this query, the ad from Kodak had very little scent aligned with intent and the ad from Circuit City is obviously just a dynamic query insertion on a generic ad. In both cases I would be expecting higher relevance in the top sponsored ads, more like what we found on Microsoft and AOL.
While I did assign relevancy ratings to the engines in this decidedly non-scientific study, I didn’t really intend this as a serious evaluation of the relevancy of the engines in question. I meant it more as an illustration of the importance of the area of greatest promise and wanted to show five examples of how the engines are currently delivering information scent in this particular area. Obviously, any type of the valuation of relevancy would have to extend over a large number of searches.
A Case for Quality Scoring
With the importance of the area of greatest promise, one can see the need for stringent quality scores on the ads that could appear in this real estate. Because this real estate is so highly scanned by such a large percentage of users, you’re really wasting a golden opportunity if you don’t take the time to match your messaging to what the user’s expecting to see. Sorry campaign managers, but dynamic query insertion on a generic ad message just won’t cut it. And the engines are seriously degrading the user experience if they don’t actively promote the ads of those advertisers to do take the time to match messaging with intent. It’s a very small triangle that we’re looking at targeting and everything within that triangle should scream relevancy to the user. Remember, relevancy will be determined on the user’s terms, not the search engine’s. The search engine that pays the most attention to this will ultimately be the winner.
Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.