Why Do People Google Google? Understanding User Data to Measure Searcher Intent

What are people looking for when they type “Google” into Google? What do they want to see when they use “eBay” as a query? How does a Google or Yahoo learn from their log files, and other user information? What does it tell them about user intent?

In an interview posted earlier today with Luke Wroblewski, Yahoo’s Principal Designer for Social Media, we’re told that the amount of user data that Yahoo has to work with while designing may be almost overwhelming. A point we haven’t seen made much by the search engines, it’s the second time today I’ve heard it. The first was during a presentation from Google senior research scientist Dan Russell.

Last Tuesday, Keri Morgret went to the December BayCHI, where Dan Russell spoke about How People Use Search Engines. She reports on some of her thoughts as well as some interesting observations about eye-tracking studies at Google. In her post, she includes a link to a video of an earlier version of the speech, How do Google searchers behave? Improving search by divining intent. The questions I started this post with are ones he asks in his presentation. He’s one of the guys who tries to find answers to questions like that.

Dan Russell joined Google from the IBM Almaden Research Center, and also worked for Xerox PARC and Apple’s Advanced Technology Group, as well as teaching at both Stanford and Santa Clara Universities. His homepage includes three sets of slides used during different versions of this presentation. The video weighs in at almost an hour-and-a-half, so if you want the shortened version, or want to read along, you may want to download one of those big PDF files.

The presentation is long, but it provides some nice glimpses into how Google works. I tried to find out when Dan Russell started working at Google when he give the presentation at Stanford linked to above, but it wasn’t easy to locate a date. It wasn’t long enough for him to not discuss some of the insights he had when he first started looking at searcher intent while at Google. His thoughts:

1. Intuitions are terrible when trying to figure out what people are searching for.

2. In particular, your intuitions are terrible.

3. That’s why Google does studies.

4. Fallacy 1: “I do it this way” so others do, too.

5. Fallacy 2 “My Mom does it this way” so others do, too.

6. Deep truth: You are statistically insignificant.

7. Deeper truth: (As a computer scientist/student/audience member) you are a couple of sigma from the norm

8. So are your friends

Understanding User Behavior and Types of Queries

While Google focuses upon search as its core mission, we’re told that the effort is of little use if we “can’t understand what the question is.” What are users looking for? That’s true not only for organic search, but also for things like images, Google Earth, print, and video, which don’t have the benefit of pagerank and link structure to index. Most of these services are in beta, and they can get it wrong now. Applications can be changed on a very quick basis, or as Dan Russell calls it, rapid prototyping on a delivery model. Especially if they can look at the usage data of those services and somehow understand it.

An example is Google Video, which at first provided the most popular results, with snippets. But it wasn’t getting the clickthroughs that they expected. So they quickly changed it to a richer display and the clickthroughs increased dramatically.

Another example was Google Maps – A couple of months after starting, the links were on the right side. After looking at log files, they decided to move them to the left, use a larger font, and add a tab for more details. They found that user behavior is influenced by small changes – and saw significantly differences in clickthrough rates for things like minor size changes in fonts. We’re told that measuring millions of clickthroughs provides interesting results.

Another issue arises when making changes to something like Maps Local. Those changes need to be echoed in places like Maps Local for Mobile. Another challenge that they face is training a culture to use a new interface – smaller changes are easier to get people used to using them.

It’s not unusual to see queries broken down into three different types, as described by Andrei Broder in his paper, A taxonomy of web search. We get to see some percentages, and a greater breakdown of query types in this presentation describing what people are searching for:

Navigational – 15 %

Transactional – 22% - Obtain 8% - Interact 6% - Entertain 4% - Download 4%

Informational – 63% - List 3% - Locate 24% - Advice 2% - Undirected 31% - Directed 3%

Search Patterns

What patterns might emerge when people search?

We’re told that it is usually a two step process:

1. Searchers find a good site, and; 2. Look for information there.

Another strategy is teleporting, or going directly to somewhere else.

The reasons for teleporting:

- Users don’t realize they can search directly for the information - Difficulties in formulating a query - The user trusts the source that they are going to

Presumably, that two step process can be a good strategy if you know something about the resource, use its search engine if it has one, and understand the structure of that site. A video of a user session is shown at this point, illustrating someone exploring a site search, snippets, and the possibility of refining their query back at the search engine.

Information about Query Sessions

Instead of just looking at individual searches, considering user sessions are an important part of the analysis of searches.

How often do people do query reformation during a user session, and what do they do when they reform those queries?

1. Spell correct helps lots of people, and shortens their sessions. 2. People often make a minor change to a word, or add a word, which may not provide the best results (people often get stuck in inefficient queries, and don’t change those much).

Some other things that they see:

1. The more words used in a query, the longer sessions tend to last. Their assumption here is that more sophisticated queries involve people spending more time searching.

2. People use longer sessions on weekends.

3. The longer the sessions, the more often they see multi-tasking (multiple search subjects in a session) and interruptions.

Advanced Searchers

Advanced searchers make up an extremely tiny fraction of the folks who search. Some of the characteristics of an advanced searcher are that they:

Have lots of meta-knowledge about content and sites Take notes (on the machine, or paper, or bookmarks) Try alternative word sequences Use quotations correctly

While they may take advantage of these things, they don’t do a lot of it.

The Challenges of Analysis of User Data

I started this post mentioning the problem of having an almost overwhelming amount of user data. Dan Russell shifts gears at this point of the presentation, and starts talking about how to analyze and reduce that data to manageable levels, so that instead of relying upon intuition, they are making meaningful use of the information.

There are two parts to meeting that challenge. The first is building a scalable data analysis system, where a portion of the data can be looked at, and parallel systems can be used to analyze the rest of the records. That type of analysis is described in a Google paper – Interpreting the Data: Parallel Analysis with Sawzall.

The second part of the analysis involves usability. The importance of field studies and lab-based usability tests of prototypes is covered in detail.

For instance, a user would perform a realistic task with a prototype, while thinking aloud. Researchers would watch to see:

- Where they have problems.

- Fail to complete a task, or take too long.

- Make an important mistake and don’t realize it.

- Misunderstand an important part of the UI.

Efforts are made to avoid helping or influencing the user, and focusing upon their actions rather than their opinions. Eyetracking is often used in this type of testing.

Field Studies consisting of interviews at the places where people actually use their computers are conducted, as well as diary and ethnographic studies.

We’re told that a large percentage of people who use search engines have very different mental images of how search engines work than people who work on search engines. An analogy used – someone opens the hood of your car, and points out a part, and asks you what it does. How likely are you to know?

The presentation describes a number of the issues they see when conducting field studies, and how they try to act upon them. The bigger issue here is how to take these types of studies and perform them in a manner which might be statistically significant. How useful might it be to get a greater sense of demographics involving different user segments and different cultures?

Some Questions and Conclusions

Some good questions that aren’t necessarily answered from this presentation:

Is one click good? Better than two clicks?

Is no click better than one click – such as when the answer is provided in the snippet?

What’s the best way to help searchers avoid distractions?

Should some diversity be mixed into results for breadth?

Why are SERPs so boring? Or are they?

How did a standard evolve across the major search engines in how search results pages look?

We’re told that Google’s focus in understanding how well they are doing in meeting searchers’ intentions has transformed from a static IR-styled analysis of query results towards longer-run, session analysis of how users interact with the search engine. This approach involves incorporating data from many kinds of studies, and using many different approaches instead of looking at a single point of data.

They don’t want to make decisions without lots of testing, they don’t want to rely upon intuition, and upon a world view centered around Silicon Valley and Stanford.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: Strategy | Stats: Search Behavior

Sponsored


About The Author: is the Director of Search Marketing for Go Fish Digital and the editor of SEO by the Sea. He has been doing SEO and web promotion since the mid-90s, and was a legal and technical administrator in the highest level trial court in Delaware.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://jambecorp.blogspot.com James

    I’m very pleased to see that Google are doing ethnographic studies, and proper usability testing. I really is a pain that the people making this products are not the typical users of them. I run into this all the time in my job, a computer person in an academic library, because I am not a librarian or (to any significant level) a library user. I constantly have to consult or at least try to think from a different perspective.

    We did some usability testing on some Institutional Repository solutions here recently and I was surprised at the number of usability issues that are so clearly identified with them. If you (I mean you in the general sense, not = Bill Slawski) have never done a usability study before, I recommend it even if just so that you can see the kinds of things and ‘average’ user struggles with… I think you will be surprised.

  • http://www.seobythesea.com Bill Slawski

    Hi James,

    I was pretty encouraged by the thoroughness, and thoughtfulness that went into Dan Russell’s (and Google’s) approach to trying to make sense of the information they were receiving from users, and that they were so concerned about collecting a wide range of information involving different methods.

    I didn’t highlight some of the issues that they came across, but one of the more interesting ones was their discovery that most people don’t understand the minus search operator “-” as something that you use to exclude certain words from results. The more common real world approach to it is that it acts like an indication of something you want to specifically include (sort of like an indication of a suffix). The example they gave was:

    wheat free -recipes

    Someone was trying to use that set of terms to find allergy free recipes which didn’t use wheat products. A lot of other searchers were using that minus operator in a similar way.

  • http://www.netexperienced.com mfshearer

    I am a believer that a fair portion of novice internet users have become so acclimated to just searching for stuff, even if they know the website url. I say this because of the web analytics at my j-o-b. Literally half of web searches to our site are iterations of our business name (we are an online university) and at least 75% of our web traffic are repeat visitors (have to go to our home page to log-in to your classroom). So people just punch in “University name here” into Google and click-through (and over 5,000 times last month they clicked on our PPC campaign). Instead of using bookmarks or typing the url direct (i don’t blame them, it is a subdomain), they use Google to get where the already know they are going. So the search engines are obviously more than just for researching and finding things you don’t yet know, but seem to be a common tool for simply getting to predetermined sites as simple as possible.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide