I believe that categorizing keywords into the finest groupings that make semantic sense is the right way to do it; often I’ll have a category with 2, 10, or perhaps 30 keywords at the most. Later, when some of the categories are turned into actual campaigns, this tight organization and relevance will tend to pay off with higher quality scores.

Since Google Adwords takes into account the relevance of keywords to the creative, obviously grouping very diverse keywords will result in low relevance, so this is why relatively fine categorization is important.

Often, however, I find myself with too many keywords to handle; even as little as 5,000 keywords broken down into 300 categories, for instance, is still not a very manageable set.

In these cases, I like to take the keyword categories and bundle the categories themselves into a *secondary* category that represents the “funnel” stage that the keyword category belongs to.

Marketers are told to think of a customer as being in one of various “funnel” stages at any given time, and even if you’re not systematic about it, you probably already think of brand terms as being “lower funnel” and research-type terms as being “upper funnel”.

Most readers are doubtless familiar with models such as “Attention-Interest-Desire-Action”, and other 4, 5, and 6 stage funnels which are pretty standard fare for marketers.

After performing my initial keyword categorization (sort of into micro-categories), I like to categorize the categories themselves into a total of *ten* funnel stages I’ve developed, which are organized around a “problem/solution” mental model.

In Figure 1, I’ve shown individual keywords belonging to each funnel stage for a variety of B-to-C funnels. Later, Figure 2 presents some B-to-B  examples.

These keywords presented could be actual keywords, but I think they are more appropriately thought of as representing *categories* of keywords:

Figure 1 - Business to Consumer Search Funnel Stages

Ten stages may seem like a lot of detail, but organizing keyword categories into these stages:

1. Forces you to really try to understand searcher’s intent.
2. Gives you a sense of where the holes in your keyword research are from a funnel perspective.
3. Resonates with clients or management and is a great way to discuss and understand a business.

For example, after going through this exercise with one client, to my great surprise, they told me that stage 2 (“Suspicion There May Be a Problem“) was almost the sole focus of their existing marketing.

Their strategy is to pull in searchers looking for help identifying their problem, establishing them early as a trusted brand in the eyes of the searcher.  This client has found that organic and offline conversions then naturally follow. Although very much a one-trick pony approach which I would not recommend for most businesses, it works great in their market.

Below is another version of the funnel with examples that are more B-to-B oriented, for those interested in that perspective;  we’ll now run through the funnel stages, explain the thinking behind each of them, and discuss which stages you should consider addressing in your marketing mix.

Figure 2 - Business to Business Search Funnel Stages

Activity Funnel Relates To

This is a very general field of activity, and will often not be a focus of marketing efforts since the customer may not actually be experiencing a problem yet.

However, display advertising that targets field-focused websites or is demographically targeted may be a useful vehicle from a branding perspective in this stage.

Suspicion That There May Be A Problem

This funnel is focused around the mental model of problem-solving; other mental models may make for useful funnels as well, but I’ve found “problems” to be universally applicable.

In this stage, there may be symptoms described but the customer does not understand the nature of the problem, or perhaps they don’t even understand that the symptoms are a problem at all.

It’s a critical stage where you can have great influence on the direction a potential customer will take; we’ll touch on this more later.

Problem Identified

This is an interesting bucket because you may have some latent versus blatant needs that you can separate out; different types of problems may actually fork off into different funnels.

Looking For Solution Alternatives

In this stage, the prospect is trying to understand the variety of approaches available to them. There are many ways to lose weight for instance; diet, exercise, portions, surgery, and so on.

This is fairly early in the research phase and can be ripe fruit for thought leadership content (great for the SEO channel as well). If you’re really lucky and you’re the only solution to a problem (perhaps you’re in a new market) then this stage may barely even exist and prospects may jump directly from stage 3 to stage 5.

Solution Space Has Been Chosen

In this stage, the prospect has decided on a particular approach for solving the problem (for instance, “dieting” to solve a weight problem).

Complicating Issues

This stage perhaps belongs alongside the funnel, but I usually place it in the middle of the research phase. Many people with problems have complicating issues; diabetes (if they are interested in weight loss), a wheelchair-bound spouse (if they are interested in travel), and so on.

Addressing these complicating issues can be a great way of differentiating your product or service and reducing friction for a final sale.

Researching A Specific Solution

Now the prospect is getting *very* specific about a particular member of the solution space (“Low-Carb Diets” in the case of a Weight Loss/Dieting funnel for instance).

Researching A Specific Brand

At this stage, the prospect is getting very serious and is educating themselves about specific providers.

Remember, brand terms are well known in the industry to convert at a higher rate as generic terms (twice the rate on average in my experience), so addressing this funnel stage should be a critical component of any online marketing effort.

Conversion Imminent

Terms that include phrases like “coupon code”, “pricing”, “cheap”, are akin to flashing red lights with a siren screaming “transaction about to occur!!!”

Spending a lot of time building out variations in this funnel section is usually well rewarded. Google Suggest is a great place to find ways that potential customers are raising their hands in these ways.

Post Conversion

Often, a neglected funnel stage, this is where you will find customers searching for things like “repairs”, “replacement parts”, “add-ons”, “upgrades”, “warranties”, and “support”.

You may or may not have offerings that address concerns in this funnel stage, but it’s important to think about them.

If you’re a travel company, trip insurance may not be something your customers will actively seek out often, and paid search campaigns targeting that concept may not be worthwhile.

If, however, your paid search keyword research turns up the concept, and you then prompt your company to put together some sort of revenue-sharing deal with a trip insurance provider to integrate their product into your cart, I would say the time spent researching funnel stage #10 was well worth it.

Which Stages Should You Target?

As most articles you’ve read on this topic probably state, you should target all of them. This is not very helpful advice though – often in marketing we have to prioritize our efforts.

If I absolutely had to prioritize the top ones to focus on initially, I would say #9, #8, #5, and #2 in that order.

Funnel Stages #8 and #9, “RESEARCHING A SPECIFIC BRAND” and “CONVERSION IMMINENT” are self-evidently critical; how are you going to leverage this great funnel if you don’t catch potential customer at the end of it?

I am, however, a big believer in avoiding cannibalization from organic search conversions, so my preference is to consider targeting competitor brand terms  before I would work on cannibalizing my own.

Funnel Stage #5, “SOLUTION SPACE HAS BEEN CHOSEN” is square in the middle of the research phase, and catches customers who are partially educated on the problem and are still early enough in the funnel to nudge in your direction.

Funnel Stage #2, “SUSPICION THERE MAY BE A PROBLEM” is important because it’s an opportunity for you to disturb the prospect’s equilibrium, a critical step in any sales process.

Much like Don Draper stated in his famous “Carousel” pitch about the term “new”, with problem defining keywords, you “create an itch, and simply put your product in there as a sort of ‘calamine lotion’”. Funnel step #2 is essentially the “itch” stage.

This stage, where the potential customer suspects but does not yet fully understand that they may have a problem, is a powerful leverage point for influencing searchers in your direction. Think of searchers as meteors, heading for earth – a slight nudge much earlier in their trajectory can have as much influence as a strong shove later in the funnel.

Conclusion

Very fine categorization of keywords can be helpful in ascertaining customer intent, organizing your efforts, and suggesting actual paid search campaigns you might run.

I have found these ten funnel stages in particular are a convenient and useful way for me to organize very large numbers of refined categories of keywords, derive insights from them, and create campaigns targeting various phases of the sales funnel.

If anyone has any other useful mental models for constructing a funnel besides the “problem/solution” approach I’ve presented here, or any thoughts on which funnel stages to prioritize and how – by all means, comment below.

Feeds define the data that may be changing regularly, like e-commerce inventory, store locations, etc. Readers may be able to generalize these terms to apply to existing rules-based tool if one is currently in use. First, a word about where this approach fits in to the broader picture.

The Broader Landscape of Managing Keywords & Negatives

There are many approaches for discovering keywords and adding them to a PPC Account, and many tools to support the various approaches. This article presents a rules-based approach and some tools that support it.

A good rules-based approach such as this often separates homegrown PPC Accounts from their more-mature competitors. This approach could be part of an overall strategy for managing Keywords and Negatives throughout the lifetime of a PPC account.

A Rules-Based Approach

A good rules-based approach is part of an overall Keywords and Negatives strategy. Most advertisers know how to manually add Keywords, many Advertisers know how to effectively use Negatives, and even How to Use Search Term View.

Quite often, efforts to add keywords in this way end up mostly manual, and occur something like: “Well, what about THIS keyword, or THAT one? Yeah, they would be good… But do don’t we have to go back and add that every where else, too?”

Good advertisers will often take that one step further and actually measure the value of their efforts by monitoring the value of those keywords over time. Some Advertisers know how to use other keyword research and generation tools such as SpyFu, Compete.com, or Wordstream.com and the like.

Some advertisers mine their SEO search queries for PPC keywords (and vice-versa), and some advertisers define rules for their automated tools that automatically add keywords based on performance (e.g.: for every broad match keyword with more than 100 clicks in a month, add a phrase match for the same keyword), or remove keywords based on lack of traffic (e.g.: remove keywords that have zero impressions after 3 months of eligibility). There are many components to managing Keywords and Negatives.

These are all healthy, productive components of a complete Keywords and Negatives strategy. A good rules-based approach helps advertisers leverage the good ideas by building them out across accounts.

• Dictionaries: Define the various groups of words that are relevant to your account
• Grammar: Define how to combine Dictionaries into actual Keywords, within your Account structure of Accounts, Campaigns, and AdGroups
• Feeds: Define keywords that change over time, as with available inventory, prices, etc.

Dictionaries

Consider a mythical PPC client, the ACME Bike Store. A previous Search Engine Land article demonstrating Stone Temple’s PPC Permutator uses a similar example. Today’s article builds on the ideas in the Permutator article. ACME’s Grammar might look like this:

That is to say, the Dictionary defines [terrain], [bike], and [product], each as a set of tokens that can be used in a Keyword Grammar to create Keywords.

Grammar

ACME’s Grammar might look like this:

That is to say, the Grammar defines meaningful ways to combine the tokens defined in the Dictionary, to create Keywords. For example, the combination of the Dictionary and Grammar so far would produce:

The list expands quickly from here. This is one of the values of this approach: Generate a lot of useful Keywords for a minimum of creative effort. However, this benefit can also be a drawback.

It is not productive to produce an excess of keywords that generate zero impressions. It can actually be counter-productive, and negatively impact Quality Score. Managing Zero Impression Keywords and Quality Score are both integral parts of any complete PPC effort, and beyond the scope of this entry.

Regardless, there is no need to over-tax those processes by wantonly adding keywords.

Grammar With Match Types

A Grammar can also define the Match Types for each Grammar entry. By defining the Match Types for each entry, the Advertiser can avoid repeating the entry for each Match Type wanted. The above Grammar definition might now look like this:

Advertisers should choose the right combination of Match Types to create a list that is full of keywords that are specific enough, but not so specific that the Grammar generates an excess of Keywords with zero traffic.

That brings up another good practice beyond the scope of this article, which would be to devise a system for automatically expanding Keywords on new Match Types when they get enough search volume. Another day…

Grammar With Modified Broad Match Support

Advertisers may choose to support “Modified”, as in the example below. One way to do that is to include the “+” sign wherever it might apply in the Grammar definition for the Keyword. Keywords being added as “Modified” retain the “+” in the output, and any output for other Match Types simply remove them.

This way, the Grammar is expressive enough to address the requirements of “Modified”, without requiring additional entries. That is one worthy goal when designing these tools; minimize the amount of effort required of the Advertiser, by maximizing the expressive power and flexibility of the Grammar Syntax.

For the sake of simplicity, the remainder of this article will use examples without the “+”.

Grammar With Negative Match Types

Similarly, a Grammar can define Negative Keywords and Negative Match Types. This helps to craft Airtight AdGroups, which is a highly recommended best practice.

For example, an Airtight AdGroup approach would prevent searches for “mountain bike” from matching for the AdGroup containing the Keyword “mountain bike tires.” To handle that case, add a Negative Exact Match “mountain bike” to the “mountain bike tires” AdGroup.

Grammar With Account Structure

It will be necessary to specify where to put all of these Keywords in the PPC Account. This is one of the final pieces needed for a complete Keyword Grammar: Account, Campaign, and AdGroup.

In addition to naming the structure elements, using a good naming convention will be valuable for many reasons; one convention to consider would be to name the group from the Dictionary, then embed the actual token used within parenthesis.

Example: Group(token), as in the table below.

The Grammar definition above defines a new AdGroup for each token defined in the Dictionary for [terrain] and [product], e.g.:

• Terrain(mountain) Product(tire)
• Terrain(mountain) Product(tube)
• Terrain(road) Product(tire)
• etc.

Defining a naming convention enables the Advertiser to report or take action in bulk based on the name of the Account Structure elements, such as Campaign and ADGroup. Example: generate a report showing the performance for all “Product” AdGroups, vs. other AdGroups to find out if customers convert differently when they have a specific product in mind.

Advanced & Optional Considerations For Grammars

There are some optional components to include in this approach. Named Sets allow advertisers to define specific groups within the Dictionary. For example, a Grammar Entry for [product:tire], which might output the set of keywords {tire, tires}.

Similarly, advertisers could implement Named Sets with Magic Words. The “Each” Magic Word directs the Grammar to expand the Grammar Entry once for “each” token in the Dictionary. Advertisers could consider more complex rules, such as [product:singles]. This could expand one line for each of the Dictionary entries in the “product” group that are single words (vs. plurals).

One worthy enrichment is the “Other” Magic Word. This is really useful for Negatives, allowing the addition of every “Other” token the in Dictionary as a Negative. For example, add a Manufacturer(Puffy) AdGroup, and add a {Manufacturer:Other} Keyword as a Negative Phrase Match to prevent this AdGroup from picking up searches from any other Manufacturer.

Advertisers can enrich the Dictionary with whatever meta-data works in a given application, and use Syntax in the Grammar, like Named Sets and Magic Words, to consume that meta-data. The simplest version of these special cases that extend the capabilities beyond the Dictionary is the use of “raw text.”

Grammar Special Case: “Raw Text”

Grammars should be capable of using “raw text” to create names, as with “ACME Bikes” to define the Account Names in the Grammar above. Similarly, Advertisers could use “raw text” in a Keyword definition to combine with Dictionary tokens, or in even in lieu of them.

A good time to use this is when starting with an existing keyword list. Just add the current Keywords do the Grammar, and then continue to add new definitions using the Dictionary. This way the Grammar describes an entire Account structure, which can be quite valuable.

Feeds

Use Feeds to define data that changes over time, such as available inventory. For example, ACME carries Puffy, Hwinn, and sometimes Sianchi bikes. A Feed might contain the following data:

Using Feeds In A Grammar

In order for a Grammar to consume the Feed above, the tool needs to understand a Syntax to express it. One could use curly-braces, e.g.: {Manufacturer}, {Category}, etc.

If Named Sets and Magic Words are implemented, consider enabling them to work with Feed elements as well. This provides the ability to expand Feed-sourced elements in a Grammar, like so:

Separate Grammar & Feed

Separating Dictionaries and Feeds provides the flexibility to adjust one while still leveraging the other. For example, an Advertiser can add a new Bike synonym to the Dictionary.

The next time the Account is generated, the Grammar will expand the new Bike-derived keywords for everything in the feed. Another example would be to add or remove a particular Manufacturer. The next time the Account is generated, the Manufacturer would be added or removed, respectively.

One note on scalability and the underlying implementation. Depending on the application, Advertisers may choose to expand the Dictionary first, or expand the Grammar first. Either way, the results should be deduped. Consider which approach will minimize the computing resources needed to address the exponential growth of the output.

Finally, it is worth noting that a similar approach, and much of the same data, can be used to produce Ads, as well as much of the rest of the information required to build a complete PPC Account.

A similar approach works for adding Geo-Targeting, with {lat} and {lon} for each {Location} provided in the Feed, and consumed in a Campaign Grammar supporting geo-targeting settings. The approach can be expanded to include whatever is needed in a given environment.

Build Your Own, Or Use An Existing Tool

This rules-based approach to Keyword generation, using Dictionaries, Grammars, and Feeds, provides a practical means of powerfully expressing new Keyword ideas into massively scaled PPC Account buildouts. The approach was presented generically in order to provide a basis for Advertisers to incorporate the ideas into tools and processes.

There are also commercial PPC tools available that already incorporate their own versions of some of the same concepts. Kenshoo offers “Real Time Campaigns”. Enterprise Travel advertisers have been using similar home-grown systems for almost a decade now. Of course Stone Temple offers The Permutator for free, and we have our own internal tools we use during Client engagements (Disclaimer: The Author is associated with Stone Temple Consulting.)

Stone Temple has built accounts with 10s of millions of Keywords, Negatives, and Ads for a client with over 800 locations throughout the United States, and uses the same approach to manage Keywords with the changing inventories of various e-commerce clients’.

These ideas can be implemented readily using home-brew or off-the-shelf tools, or by engaging with Consultants who have access to them. On a personal note, I eagerly anticipate the day when Search Engines adopt these ideas and make them available to Advertisers.

How would a rules-based Keyword Generation approach impact your PPC efforts?

Amazing because every year, football fans flock to search engines searching for the start time, and until now, organizations like the NFL, the playing teams, and the broadcasting station didn’t show up at all in search results because none of their sites answered the question. Seem crazy?

• 2009  Results: In 2009, start-time related searches were among the most popular the morning of the game, but neither the NFL nor NBC were anywhere to be found.
• 2010 Results: In 2010, both nfl.com and cbs.com had significant technical infrastructure issues that kept search engines from crawling and indexing the content. Again, the search results were sad and this time, full of spammers trying to capitalize on the search volume.
• 2011 Results: In 2011, problems continued. But news organizations jumped in, and the Huffington Post in particular ranked well for its article that simply listed all of the various ways people were searching for the Super Bowl start time. (That article was later “edited for clarity”).

This year, things are finally getting better. Even the Huffington Post, while still getting every variation of spelling and tagging in the article for maximum search coverage (“For starters, it’s two words, not one. “Superbowl” is an incorrect spelling.”), has filled out their article a bit with actual information.

The results could still be better. While [Super Bowl start time] has overall higher search volume than [Super Bowl kick off time], the latter is the top search this morning, and NFL.com only ranks for the former (HuffPo does quite well with the latter). Superbowl.com, which redirects to the NFL site, ranks, but as I mentioned in earlier years, this domain 302 redirects to nfl.com. A 301 instead would consolidate the domains (including value signals such as links), which might cause the target URL to do better overall in relevant searches. But still, compared to earlier years, I’d call these results a win for the NFL.

Sadly, NBC, the Giants, and the Patriots, and TV Guide all fail to appear in results once again. Even though both Google Insights for Search and my articles over the years should have prepared them for this year’s search interest.

Why should these sites care about showing up for these searches? They’ve invested substantially in site content and those seeking out the game start time are a perfect audience for that content. Searchers would click for the start time and stay for the fan jam videos and view the ads.

Of  course, Super Bowl viewers will see lots of ads anyway today. But that’s a topic for the next article.

Related:

• 2009 Super Bowl Start Time
• 2010 Super Bowl Start Time
• 2011 Super Bowl Start Time
• 2009 Super Bowl Commercials
• 2010 Super Bowl Commercials
• 2011 Super Bowl Commercials

The ad network analyzed search referrals on “hundreds of millions” of impressions across sites in its network between January 9th and 12th. And the longest search referrals — at an average of 4.81 words — came from Ask.com. AOL users are at the other end of the spectrum; their user queries average barely above four words, by far the shortest of the five sites that Chitika studied.

It makes sense that this would be the case, since Ask.com has been refocusing on questions and answers in recent years — and asking questions tends to involve more words than other types of queries.

It would be great to get real search query length data directly from the search engines, but they’ve never made a habit of sharing that kind of information. In May 2010, Google did release data showing that 54.5 percent of queries are more than three words — which falls in line with the Google (and Bing) data that Chitika shows above. But that’s the last time I’m aware of that either Google or Bing shared such information.

Keyword research is your time to understand the market you are competing in and how people search. It is your opportunity to comb through the competition and learn their keywords, content and link-building strategies. It is your opportunity to map out what you should track for your website, your market competitors and your keyword competitors.

If you plan a months- or years-long relationship with a client or website, you owe several hours or days to get this right.

Keyword Research Tips

Keywords are the foundation of search engine optimization. It is all about getting traffic from relevant search queries, the keywords people use to find our products or services or whatever we offer that our target market is looking for.

Placing a handful of words into the Google Keyword Tool, exporting the results, then calling it a keyword chart is skimming; it is not keyword research. Quality keyword research takes time and investment.

• Before you open one keyword tool, study the topic you are researching, at least to the point that you can explain it intelligently to someone else and answer basic questions.
• Study your marketplace competitors. These are basic sources for seed words and phrases to put into keyword tools.
• Seeds are the words and phrases you enter into the keyword tools. Track these and use the same ones in every keyword research tool.
• Use multiple keyword research tools. Every service has its strengths and weaknesses. Using different keyword tools is like seeking differing points of view. You want to be certain you have the best information possible.
• When you have some good keyword candidates, begin studying the search engine results for rankings competition and additional keywords. In SEO, the real competitors are the websites that rank for your keyword targets, not just your marketplace competitors.
• Revisit the keyword tools. Look up additional seed keywords you may have added along the way. Create a complete dataset for every keyword research tool you use.

Keyword Selection Tips

Once you have a list of keyword candidates, you must cull through it to find your keywords. This is where a lot of people throw up their hands and give up or try to over-simplify the process.

Going back and forth between dozens of export files from different keyword tools is not practical, so I use a database to compile a master table that I later export into Excel.

Even if you are not a database wizard, anyone can learn to combine data into a table using Microsoft Access. You can do this in Excel too, though I find that more taxing.

• Compile your research into a master table so you can sort it and filter it.
• I sort my keyword candidate by the number of words in each keyword or phrase first, then by the number of searches. Here is the Excel formula I use to create a word-count column:

• Set aside or check off relevant one-, two- and three-word phrases.
• Set aside or check off embedded keywords. Before Chris Anderson coined The Long Tail, I used embedded keywords to describe longer key phrases that contained shorter keywords. Search for each relevant one-, two- and three-word keyword, then mark the longer keywords that contain the shorter keywords.

At this point, what is left will be like looking for diamonds in a trash heap. There will be lots on non-relevant words and words with too little traffic. Comprehensive research is important, but now it is time to get practical.

• Set some limits. Depending on how much traffic the website I am optimizing receives already, I will set a lower traffic limit between 100 and 1,000. The more traffic my website is getting, the higher the limit I set. Anything below the limit gets culled.
• Review each keyword candidate you have left. If it is relevant, mark it or set it aside.
• At the end, copy all the keywords you marked or set aside into one table. These are your keyword candidates.

A keyword is not a target until you assign it to a page and begin optimizing for it.

Keyword Tracking Tips

You may be tempted to toss all your keyword candidates into a ranking tracker and monitor them. I advise against this.

Keyword candidates are not the same as keyword targets. Any keyword you have not matched to a specific web page will just be a distraction.

Reports filled with unassigned keywords tend to go unused. Also, if your boss or your client sees a bunch of keywords that are not on their website or for which the rankings and traffic are low, it will open you up to uncomfortable questions. Don’t put yourself in a position where you have to say, “Oh, just ignore those.”

What should you track? In my book the most important measurements are:

• The number of organic visitors that each assigned target keyword brings from each search engine.
  • Exact Match
  • Phrase Match
• The total number of organic search visits.
• Keyword diversity: the number of different keywords bringing traffic to the website from each search engine.

I still track rankings for high-tail keywords, grudgingly, once I assign them to a specific URL and start optimizing for that keyword. Actual search engine rankings are difficult to track. Too many different things influence the rankings.

Universal search result inserts (news, images, video, etc.), local search results, Query Deserves Freshness, Query Deserves Diversity and other factors all change the contents and display of the organic listings.

If I search for coffee house in Seattle and then in New York, I will get two completely different sets of results. Track traffic as your key performance indicator and use rankings as an imperfect estimate to determine things you cannot measure directly, like how much traffic your competitors may be receiving for the same keywords.

What About Long Tail Keywords?

If you are pursuing a long-tail content strategy, you may want to add rankings for recently used long-tails to your reports, but I would remove older long-tails from your reports as you add new keywords and content.

Unlike high-tail SEO, long-tail optimization tends to be “set it and forget it.” If you are not actively working on optimizing a keyword, it probably should not be on your reports.

A good alternative to traditional ranking reports are the search query reports in Google’s and Bing’s webmaster tools. These show average rank, clicks and impressions. This data is now in Google Analytics as well under Reports » Traffic Sources » Search Engine Optimization.

I like having clicks and impressions because, if a targeted keyword gets lots of impressions but few clicks, either the listing needs work or the keyword is a poor target. The downside is that these offer no competitor data.

Keyword Assignment Tips

It is important to be realistic when assigning keywords to URLs as search engine optimization targets.

• Understand keyword difficulty. Blindly optimizing for the most heavily searched keywords can lower non-paid search traffic.  When a website has nowhere near enough authority to earn a top ranking for a high-volume keyword, but optimizes its most linked-to pages for that same keyword, this may steal an opportunity to optimize for a more realistic keyword.

Be honest in your perspective.

• Use embedded keywords. Optimizing a page for a medium-tail keyword that also contains a high-tail that can get rankings and traffic now, while the page earns the authority to rank for the high-tail.

A medium-tail keyword is defined by whether you can successfully compete for a ranking as much as the number of search queries it receives.

• Use long-tails to support mid-tail and high-tail keywords. Not every page can be an SEO hub page — a page targeted for a specific, competitive keyword. The classic content strategy is to create lots of pages for long-tail content. As you do this, try to create content related to the higher-tail keyword hub pages and link from your articles — using optimized anchor text — to those pages.
• Do not try to optimize every page for a keyword. Every website has pages that will never receive search engine traffic. For example, a webpage showing a process chart may be great for people but thin content for web search. Think carefully before assigning a decent keyword on such a page.
• Consider searcher intent. People may search for information or entertainment or acquisition. Don’t optimize a sales page for an information keyword. Try to match page content and keyword assignments with intent. Start by looking at what types of pages already rank.

At the start, I wrote that we owe it to our websites and clients to get keyword research right. With all the research and reporting tools out these days it is easy to spend too little time preparing or researching then attempt to track more keywords and metrics than we can ever use.

Invest the time and thoughtfulness up front. Select which keywords you will optimize for. Choose pages for those keywords. Remove distractions. Do this and you will always focus on things that matter.

Google Correlate Added To Google Trends

Google Correlate has survived the chopping block. Previously available at correlate.googlelabs.com, it’s now been given a permanent home as part of Google Trends and can be found at www.google.com/trends/correlate.

Correlate has been described as “Google Trends in reverse.” With Trends, you provide a search term and get back data related to the term. With Correlate, you provide the data first. Here’s how Vanessa Fox described it just a couple months ago in our article, Google Correlate: A New Way To Research Keyword Popularity & Trends:

With Google Correlate, you can upload data charted over either time or space and Google will look for matching patterns in search volumes. If you don’t have data of your own to upload, you can simply specify search terms, and Google will calculate the trending pattern and show matching patterns.

Google Sets Shutting Down

Google Sets is another keyword-related tool, but it’s headed out to pasture. The latest update on Google Labs’ website says Sets will be shut down by September 5th. It was a simple tool that could identify and predict groups of related keywords from a small list (of up to five items). It was discussed in these previous articles here on Search Engine Land:

• Google Sets & Squared: Powerful Keyword Research Tools by Tony Soric
• 7 Incredibly Valuable But Underused Free Tools For PPC Marketers by Brad Geddes
• The Giant List Of Keyword Tools by Josh Dreller

More Google Labs Decisions

Those are just two of several announcements added to the Google Labs website over the past few days. Only one other experiment is specifically listed as being phased out: Places Directory, an app that launched in 2009 but has been essentially replicated by many of the features that are now standard in Google Maps for mobile.

Several other Labs projects carry a new notation that says, Although Google Labs is winding down, (NAME OF PROJECT) is available as usual on Android Market. That list includes:

• Google Shopper
• Open Spot
• Intersection Explorer
• WalkyTalky
• My Tracks for Android
• Google Goggles
• Sky Map for Android
• Gesture Search
• Finance for Android
• Google Listen

It’s not clear if Google is saying those projects will remain available in the Android Market after Labs shuts down, and when asked, a Google spokesperson would only say, “We don’t have any future plans to announce at this time.”

Three other updates: Google says that Web Application Exploits and Defenses can be found at google-gruyere.appspot.com. Scripting Layer for Android can be found at code.google.com/p/android-scripting. And Google Earth Engine is being moved to www.google.org/earthengine.

With this latest round of announcements, Google has now decided the fate of 28 of the 53 existing projects in Google Labs. See the “related articles” below if you missed the previous updates. The company isn’t saying what the timeframe is for determining the future of the remaining projects. But they’ve decided on just over half in about five weeks, so that may suggest it’ll be another month or so before Google Labs formally closes its doors.

Surprisingly, Facebook had a larger percentage of searchers over the age of 35, while Google+ trended  younger. 34.2% of searchers looking for Google+ were under 35 years old and only 24% of Facebook searchers were under 35. In addition to the younger demographic, Google+ searchers skewed towards higher incomes. Facebook saw 22.8% of searchers with an income of above $100,000 a year, while Google+ had a 32.1% makeup.

ComScore also elaborated on the information used in this report:

Each example analyzes the demographics of searchers that used the branded terms for the month of July, and are based on head of household. The index baselines are the searcher demographics for the entire US search population. Available measures are Age, Income, Location (home/work), Region of the US, Household Size, and Presence of Children in the Household.

For more information, see the full comScore demographic report.

In my last column, I started to make the case for how SEO changes when targeting mobile searchers, starting with the 14 differences between desktop and smartphone search results in Google that I was able to spot easily.

Today, I’m going to focus on keyword targeting, and how mobile searches could be affecting your bottom line today.

Searches Without Keywords

First, a scenario to consider: a business traveler is having a beer in the hotel bar after the first day of a conference.

She likes to try local beers when she’s in a new city, and this summer evening in Chicago calls for something light. She asks the bartender for a local summer beer and he gives her Goose Island Summertime.

She likes the beer, and wants to know more about it, but at this point the bartender is at the other end of the bar and she’s about ready to leave, so she takes out her Android phone and scans the label with Google Goggles.

There is no direct match for the type of beer, but Google does find a logo for the company that makes it, and gives general web results for the keyword [goose island]:

Our business traveler is able to click on the Goose Island website, and find out more about the beer, but with the current result set and the desktop website in the first position, it might take her two or three times longer than it should to find out more about the beer, and it’s impossible for her to recommend it to her friends on Facebook, or have a case of it shipped to her back home.

Given that she’s about ready to leave anyway, it’s unlikely that she’ll have the patience to complete her search session satisfactorily and even less likely that she’ll be able to remember the beer tomorrow morning.

I bring this up because this is not some futuristic scenario that will be possible in a couple of years. This is mobile search today. It’s also something that can be optimized for by the brand owner.

Furthermore — and this is the crucial difference for how marketers need to think about keyword research with respect to mobile— no keyword was ever considered or directly entered by the searcher.

Does this mean that we won’t need keyword research in a mobile visual search world?

Absolutely not. If you look at what the search engine is doing, you can see that it is scouring its image collection for related images and suggesting keywords that are related to the image, based entirely on what keywords the image is optimized for. Keyword research is still necessary, but there’s a different use case, and a different results set to optimize for.

Luckily for the Goose Island brewery, someone had optimized their logo for the phrase [Goose Island Logo], so Google was able to match the image to the keyword and provide somewhat relevant results.

However, if marketers from Goose Island had considered this use case when optimizing the website, they could have made sure that all of their logos for all of their beers were clearly displayed and optimized for logo queries in Google Image Search, and that each of these logos was attached to a mobile site that allowed the viewer to recommend their brand on Facebook and have a case shipped to their house in as few steps as possible.

Unfortunately, this wasn’t the case, and this excellent craft brewery in Chicago missed a chance to grow a little more.

Are you considering mobile visual search when you’re doing keyword research and optimizing for logo queries? I would guess most people aren’t, since the use case is currently rare; but it’s only one mobile scenario of many that could affect brands that most aren’t even thinking about today.

Context Changes Meaning Of Queries

Google released a study in late April 2011 that details the shopping habits of smartphone users, and what they found should be changing how you perform keyword research.

I remember when I first learned to keyword research back in the early 2000s, I was asked what keywords I would type into Google if I were searching for a specific topic. The point was to demonstrate how some searchers use different queries than the ones you use, and you have to think about all variants in order to find the most qualified keywords that will bring searchers to relevant content on a brand’s website. In the early 2000′s, everyone was searching in a browser on a desktop computer, so there was no need to give context in order to understand user intent.

Today, you wouldn’t be able to answer that question of user intent without first understanding the user’s context. For example, according to Google’s research, 59% of smartphone users report using the mobile Internet while waiting in line, 48% report using it while eating, and 44% report using it while shopping.

If we were trying to research certain concepts prior to the surge of the mobile Internet, like how these users searched for coupons related to a brand, we would focus on optimizing a web page for variants of coupon and sale terms until we optimized a page intended for desktop browsers with printable coupons and coupon codes on it.

However, if we were to optimize that same page today, given that most mobile users don’t have the ability to print coupons, and some have the ability to scan them on their phone, understanding the mobile context both provides additional keywords, and negates keywords that we may have used for a desktop Internet-only page.

As a result, if a marketer really wants to optimize a page for coupon keywords in this age of mobile searches, they should either optimize a desktop landing page for all keywords and include both desktop and mobile keywords in a desktop user experience (good), or include mobile keywords and a mobile coupon in a mobile user experience and desktop keywords and printable coupons in a desktop user experience (better).

To ignore mobile searches entirely in a world in which 1 in 7 searches on Google are performed on a mobile device, and as many as 30% in the restaurant category, is a sure way to frustrate users and lose business.

Fortunately at this point, Google’s keyword tool provides mobile keywords and volume for feature phones, as well as smartphones and tablets, in addition to the desktop volume that they’ve always provided, so savvy marketers who want to understand how context changes keywords for their business finally can get data to help them do that. Hopefully this article, along with Google’s research on smartphone users, is enough to demonstrate that keyword research is changing, and those of us who do it regularly need to catch up before it’s too late.

Having explored the ways in which mobile searchers are changing the keyword research game and the ways in which mobile search results differ from desktop results, I’ll spend my next column detailing mobile’s effect on links and linkbuilding.

In the Just Behave column, we’ve discussed information architecture as it is used for search engine marketing and usability. One of the points mentioned concerns the gathering of terms and words that are used for taxonomies, link labels, category setup, navigation labels and content development.

If you’re only interested in keyword research for page rank purposes, you’re missing out on what people really care about. Since Google and Bing have learned how to determine what people like the most, it make sense to look for new ways to get that information for your web site.

Social Awareness Streams

A recent paper called “Hip and Trendy: Characterizing Emerging Trends on Twitter” calls social awareness streams “a class of communication and information platforms”. Those platforms are social networking sites such as Twitter, Facebook and even Youtube. Any place that we post 140 word comments with or without links, comments, pictures, videos and links are social online water coolers.

We’ve learned to Twitter while watching our favorite TV shows or mourn together the death of famous people. Facebook is used for both personal connections, as well as business use for marketing and gaining brand recognition through the involvement of “friends” and “fans”.

A slew of studies have shown the global impact on information, communication and the media due to popular social networking websites. For example, information and news are instant. In real-time using Twitter alone, hundreds of millions of users can log in and learn the latest interests, happenings, events, news and even public attitudes and opinions.

Sure, there’s always a debate about whether all this access to information is healthy or even necessary. But in general, the world has adapted and certain technologies thrive on this constant instant access to us. For web design, online marketing, user experience design and content writing, social awareness streams offer the opportunity to discover trending topics, opinions, and new resources. Your target market is talking to you and all you have to do is tap into their discussion streams.

How To Find New Topic Angles to Write About

If you’re accustomed to doing all your writing research using books or search engines, you may love trying Twitter. In the example below, I want to write new content for cooking websites that are looking for new recipe ideas. While I have favorite websites that can give me good information, I would rather look to see what’s popular with the world, just to get a new angle or perhaps learn something new.

Twitter has a search field at the top. Twitter users use what’s known as “hashtags” to help categorize and sort their “tweets”. It’s not a requirement to add them but if a tweet is expected to be found in searches for specific topics, it’s helpful. For this example, I chose the hashtag “#recipes” to see what came up.

As you can see, there are a lot of information choices with 71 more tweets waiting. In the upper right, are suggestions for me to “Follow”. These aren’t paid spots. Twitter displays what its algorithm determines may be the best choices. The tweets themselves are a running scrollable dialog. They’ve bold faced my hashtag and as you can see, some posts have additional qualifiers, such as “#breakfast.” Sorting options are offered such as tweets with links to resources (sites, articles, blog posts, pictures) and also local tweets.

Twitter is helpful for offering new ideas. Who knew there was a site with 17,000 cooking videos? And of course, with the Royal Wedding happening, any site related to recipes, weddings and food can take advantage for promotional purposes.

For keyword research, new combinations of words can be uncovered in Twitter, such as “royal wedding scones” and “yoga and tea” as a combination or long tail idea. When you spend enough time browsing specific hashtags, you’ll begin to see the reoccurrence of certain words or discover popular themes and topics. It’s a fantastic way to wipe away writers’ block or create new blog post titles.

Detecting Trends

A trend on Twitter is the easiest to find as opposed to Facebook or search engines. Facebook has what I call “mini-events” that are limited to certain groupings of people. For example, when I went to my high school class reunion, I posted pictures afterward in my Facebook account.

Not only did the people who attended who were on Facebook leave feedback, but friends from other years responded with comments too. The strange part is that this event was held publicly to all my “friends”, some of which are business colleagues and family members.

Facebook may not instantly expose global trends, but regular posting offers some surprising results. One of my high school classmates developed a new interest in website design just because she read some of the articles and blog posts I had linked to. Sometimes you can figure out what’s popular with people depending on their responses to posts, such as anything by the Dalai Lama, politicians and community sites.

Manually monitoring Facebook takes time but you can discover who the frequent posters are in your “Friends” circle and get an idea about who generates the most discussion. I have a few friends who love political debates. The discussions give me another perspective that news media sites don’t because there are no slant or propaganda leanings.

Twitter has grown popular for uncovering the latest trends based on how many discussions there are on a topic. It’s so popular that users know they can drive up the number of tweets to create a trend if they wanted to. Most of the time a quick look at Twitter trends just raises our curiosity.

For example, I found “Baskin Robbins” trending while randomly playing with a search for “Search Engine Land”. As you can see, the trends don’t need to be related to your search terms. And of course, Twitter is smart enough to know a search for Search Engine Land should also point to @DannySullivan!

Clicking on the trending “Baskin Robbins” led to results that provided the news (and feelings about) a 31-cent night promotion. Had it not trended, many people might not have known about this limited time special.

Tweets on a Trending Topic

Collecting data on trends is the focus of many studies, including ways to create new software that can deliver taxonomy information based on the trending data. Twitter computes trends on an hourly basis (how they do is not made public.)

The information can be used to understand geographical trends and even uncover areas of the world where certain information is more popular than others. (You can search by location in Twitter trends.) Tracking social awareness streams helps us understand a trends origin and context. There’s a difference found between informational/news trends and conversational. Who retweets more? Are local events tweeted less than national events?

An end goal of some of the SAS studies is improving ranking, prioritization and filtering of extracted data. Automated tools can be developed for individuals, organizations and communities wishing to collect and use social awareness streams. Many new ideas are being experimented with.

One experiment from my hometown newspaper is the creation of Tweeter feeds targeting the local community and also inviting the community to tweet, retweet and discuss the items. Even the newspaper publisher is available to tweet with. The result was to get more intimate insight into what the locals care about, which in turn gave them new ideas for stories, columns and new bloggers.

Google News: A Copy Editor’s Nightmare

Consider this page from Google News, the “Muammar al-Gaddafi” page that pulls in headlines from all over the world about the leader:

That page would give any newspaper copy editor fits, given that there are at least four different spellings showing up:

• al-Gaddafi
• Gaddafi
• al-Qaddafi
• Qaddafi

That’s just for the leader’s last name. His first name is spelled in at least two different ways, on that page.

A Name By Any Other Spelling Doesn’t Search As Sweet

Now consider people who keyword search for news. If they search for kadhafi, they get these headlines:

Now search for exactly the same person, only with a different spelling of gaddafi, and you get a different set of results, lead by different publications:

Notice that quote at the top of the page? That quote page again reflects the lack of unity on spelling Gaddafi’s news in the journalism world:

Survey Says: News Organizations Disagree

I checked several major news publications in the United States, as well as the UK, Canada and some for India and Japan. You can see the lack of agreement, even among the three major wire services, AP, AFP and Reuters, which I also checked:

Gaddafi: Reuters (Wire), Washington Post (US), Huffington Post (US),  The Guardian (UK), The Telegraph (UK), The Times (UK), The Mirror (UK), The Sun (UK), The Mail (UK), Bild (Germany), Times Of India (India), Daily Yomiuri (Japan). Asahi Shimbun (Japan)

Gadhafi: Associated Press (Wire), Wall Street Journal (US), USA Today (US), San Jose Mercury News (US), Chicago Tribune (US), Toronto Star (Canada), The Globe & Mail (Canada)

Kadafi: Los Angeles Times (US)

Khadafy: New York Daily News (US) & New York Post (US)

Kadhafi: AFP (Wire)

Qaddafi: New York Times (US)

At first glance, most publications seem to go with “Gaddafi,” but that’s largely due to the UK. Every major UK publication I checked used that spelling, unlike the disagreement among US papers and the wire services. Two major Canadian papers went with “Gadhafi.”

Even Gaddafi Doesn’t Say What’s Right

Who’s right? An interesting AP story suggests that the AP is writing the name in English as Gadhafi (the AP spelling) would prefer:

The Associated Press goes with Gadhafi. Why? It has to do with pronunciation – along with a series of letters the Libyan leader sent to American schoolchildren more than 25 years ago….

How does Gadhafi himself pronounce it? That’s easy since he refers to himself in third person quite often. He tends to say “Gath-thafi” with the middle letter pronounced like the soft “th” in “either.”….

Flash back to 1986, a year that started out with the AP (and many others) spelling the Libyan leader’s name Khadafy, based on the advice of Middle East experts. That changed when he sent letters to American schoolchildren, signed in Arabic script over his typed name: Colonel Moammar El-Gadhafi.

The story goes on to say that, however, the AP also chooses to drop the definitive marker of “El” from names, which is why it ends up with “Gadhafi” — which isn’t, in the end, how he seems to write it.

Then again, The Reid Report does a fantastic round-up on how on what’s apparently Gaddafi’s official site, he goes with “Al Gathafi” there. That round-up also covers a number of spellings used in 2002, where the Libyan Broadcasting Corporation disagrees with Gaddafi’s official site as well as the Libyan embassy — not to mention the US White House.

The Library Of Congress & SEO From 1986

It also points to a 1986 article from The Straight Dope which found at least 12 different spellings used by major media outlets. It lists what apparently the US Library of Congress had as guidelines on his name, which makes me think the library must have been employing an SEO way back before we had search engine optimization:

For the record, here’s the official Library of Congress rundown on how to spell ol’ whatsisname:

(1) Muammar Qaddafi, (2) Mo’ammar Gadhafi, (3) Muammar Kaddafi, (4) Muammar Qadhafi, (5) Moammar El Kadhafi, (6) Muammar Gadafi, (7) Mu’ammar al-Qadafi, (8) Moamer El Kazzafi, (9) Moamar al-Gaddafi, (10) Mu’ammar Al Qathafi, (11) Muammar Al Qathafi, (12) Mo’ammar el-Gadhafi, (13) Moamar El Kadhafi, (14) Muammar al-Qadhafi, (15) Mu’ammar al-Qadhdhafi, (16) Mu’ammar Qadafi, (17) Moamar Gaddafi, (18) Mu’ammar Qadhdhafi, (19) Muammar Khaddafi, (20) Muammar al-Khaddafi, (21) Mu’amar al-Kadafi, (22) Muammar Ghaddafy, (23) Muammar Ghadafi, (24) Muammar Ghaddafi, (25) Muamar Kaddafi, (26) Muammar Quathafi, (27) Muammar Gheddafi, (28) Muamar Al-Kaddafi, (29) Moammar Khadafy, (30) Moammar Qudhafi, (31) Mu’ammar al-Qaddafi, (32) Mulazim Awwal Mu’ammar Muhammad Abu Minyar al-Qadhafi.

SNL From 1981 On News Confusion

But wait, there’s even more. Looking further back, Tim Ruder from Perfect Market got big laughs at our recent SMX West conference, when he showed a clip from Saturday Night Live in 1981 that was made fun of the issue:

The clip itself isn’t online that I can easily find right now for free (it’s from Season 7, Episode 8 for those with access via Netflix), but Perfect Market posted a transcript here (and the opening image for this story comes from that post). The transcript begins:

This man, Libyan leader, Col. Moammar Kadaffi, has been the study of intense news coverage this week by every major news organization in America. However, every time his name appears in print, it has a different spelling.

How Do People Search For Gaddafi?

If Gaddafi himself doesn’t make clear how people should spell his name in non-Arabic alphabets, and the news publications can’t agree, can we crowdsource a solution? How are people searching for the man?

Here, we have more agreement. The chart below is from Google Trends, which shows how people are searching from around the world. I checked on the six spellings used by the major publications above, to see which were the most searched for over the past 30 days:

You can only look at five terms at a time on Google Trends, but I quickly determined by running various combinations that “khadafy” is the least popular on the list. As for the others, “gaddafi” was the most popular spelling used by searchers, followed by “kadhafi” and the rest.

Search Popularity Says: “Gaddafi”

The chart above shows this via trend lines — “gaddafi” is well above the others, over the past 30 days. Above the trend lines, you can also see little bar charts that reflect popularity, which I’ve enlarged over to the right.

The numbers are an index value, based off the most popular term, “gaddafi.” They tell you in relation to that, “kadhafi” is only 44% as popular, “kadafi” is 34% as popular, “gadhafi” is 20% popular and “qaddafi” is way back at 8% as popular.

News Sites Ignore What Searchers Seek

What’s especially interesting is the lower line chart. This is “news reference” volume, and it shows how often each word is used in news stories, rather than how popular the term is based on searches.

You can see that the second most popular term is “gadhafi” as written in news stories that Google has collected, despite that being the fourth most popular term that people actually use. That’s a disconnect that some news publications may wish to reconsider.

News Search Also Says: “Gaddafi”

Indeed, using another tool, Google Insights For Search, you can see what terms are most popular specifically among those doing news searches.

Below is a chart showing the trend over the past seven days, and I’ve tucked the key with some index values on the side (note, the chart was set to seven days, but Google Insights only reported five days of data, for some reason):

This is similar to what we saw for the data from general searches — “gaddafi” is above the rest, though for a few days, “kadhafi” was more popular. If you go out further, sometimes “gaddafi” leads (as over the last 90 days or over the past three years), while sometimes “kadhafi” does (as over the past year).

Popularity Can Vary By Country

Location also matters. In the United States, it’s “gaddafi” far more than other spellings, according to Google Insights:

The same was true when I did spot checks for the UK and Canada. But in France, it’s “kadhafi” that leads:

Influence Of Google’s Own Suggestions

Finally, could Google itself be influencing how people search? Certainly, given that it will suggest terms. Those are based on how most people search, of course, but they can cause people to shift.

Consider someone who might think to search for “gadhafi,” and what they get, as they start typing that in:

See how Google suggests that they search for “gaddafi” — that’s going to cause some of them to select this option, regardless of what they originally started typing.

However, it doesn’t work this way for “kadhafi,” which is the second most popular search term. Start typing that in, and you get this:

In this case, there’s nothing that pushes people to instead complete a search for “kadhafi.”

Meanwhile, at Google News, typing in “kad…” causes the suggest box — which uses different data than at regular Google — to list “kadafi” as the top term along with “kadhafi” and “gaddafi” as suggestions:

Should Your Spelling Style Be Based On Search?

The takeaway from all this? I don’t think the decades of confusion will suddenly get resolved. However, if I were a news publication, I’d be using tools like these above to get a better idea about how my particular audience is searching for Gaddafi and using the spelling that makes the most sense based on that.

Sorry, New York Times — going with “Qaddafi” probably makes little sense. But at least you’re better than the New York Daily News and the New York Post going with the even less popular “khadafy.”

Postscript: Doug Edwards points me to a West Wing episode where White House chief of staff Leo McGarry takes issue with the New York Times and the spelling it uses, a least in relation to its crossword puzzle. Here’s the clip:

(featured home page image via Shutterstock.com)