Seeking The Holy Grail Of The Long Tail

Finding methods to chase the long tail of search—those less common, non-popular keywords that can nonetheless be worth their weight in gold—is kind of like seeking a holy grail. They are not easy to find, but it has a big payoff (presumably) when you find them.

If you have been involved in SEO for any length of time it is quite likely that you have heard a lot of talk about the long tail. This is that well-known phenemenon where the high volume search terms for a given market space (the ones that everyone chases) represent only 30% of the potential keyword volume for that space.

Why? Because there are no fixed rules for searching. People simply enter what comes into their minds. They don’t worry about grammar. They don’t worry about spelling. And many searchers don’t really know how the search engines work, so their queries can be all over the map. Another interesting trend is that queries are getting increasingly complex, with people using multi-word phrases rather than just one or two keywords.

For example, in September 2008 Google indicated that their average search query length exceeds 4 words. Equally interesting was Udi Manber’s statement at Google’s Searchology event in May 2007 that 20 to 25% of the queries Google sees in any given day are search queries that Google is seeing for the first time ever. OK, so the long tail is big. How do we chase it without chasing our tails?

The problem with keyword research tools

To find long tail terms, you might be tempted to pull up your favorite keyword tool, such as Wordtracker or Keyword Discovery and start looking for those long tail terms. But there are two problems with this idea:

The first problem is that long tail search terms usually occur infrequently—once a week or even less frequently.

The second problem is that the data sample that keyword tools look at is very thin. They may have access to a few percent of the search volume taking place on the web.

Together this means that the keyword tool can not easily find long tail terms because the incoming data (long tail search queries) is sparse, and the keyword tool’s sample set is small.

Keyword tools can still help solve the problem

Note that we said that the keyword research tools could not easily be used to find long tail terms, but they can still be useful. To illustrate, let’s go straight to an example.

Imagine that you own a nationwide rental car company, and you want to figure out the long tail terms for search terms such as boston rental cars. The hard part is that the search volume reported by Wordtracker on such terms is quite small, so it is hard to get a lot of data on where to go with this. How do we solve this problem? By using the keyword tool to increase the statistical significance of its own data.

OK, so I tried to make that sound a bit mysterious. But, here is how it works. Start by searching on Boston rental cars, and you get something like this:

Then do Seattle:

Then try Miami:

For purposes of our example, we will stop there, but the more cities you enter the better. Note that there is no need to limit the cities you search on in this fashion to ones where your rental car business has locations.

Interpreting our long tail data

Now that you have done the raw research, it is time to interpret the data. Start by putting all your results in a single list. What we are looking for are unusual modifiers that could apply to any of the cities where you are located.

For that reason, you can strip out the words “rental” and “car” from all your data. You can also remove overly geographic specific things that won’t apply to other cities, such as references to “provincetown”, since that is pretty specific to Massachusetts, or brand names such as “avis” or “budget”. While you are at it remove any stopwords, such as “the” and “in.” When you are done remove any nonsensical remaining entries, and you should end up with a list similar to this one:

• last minute airport
• suv
• cheap
• luxury
• specials
• airport
• ferrari
• sports
• best rate

Based on the cars you carry, you can remove other things that don’t fit. What you are left with is a list of typical keyword modifiers for the basic search phrase rental car. Even though these terms only appeared in one city each, there is no reason you can’t incorporate them into your PPC keyword list for all cities, and/or your on page SEO for all cities. For example, the term sports car showed up only in the Boston list, but it is a safe bet that there are people in Miami and Seattle who like to drive fast also.

The reason this works is that you have used the data across multiple queries to increase the statistical significance of the data that the keyword tool has available. While there may only be a hundred or so related queries for one city by itself, you can turn this into an analysis of thousands of queries by looking at multiple cities.