In my last Just Behave article, I shared a set of metrics that will help you create a powerful SEO business case for your natural search projects.

Through the process we’re now able to tell the CFO something like this: “Our addressable market spans 200,000 unique keyword markets and totals roughly 5,000,000 searches per month, of which we’re capturing 1% currently. The cost of acquiring that traffic through PPC totals $1,000,000 per month.”

That paints a compelling (perhaps rosy) picture. But what’s driving your current performance? Why do you believe you can do better? In this installment, I share three more metrics that can you help create strong baselines, set realistic expectations, and direct your campaign towards the most effective outcome.

Keyword reach

So you’re getting 50,000 searchers from natural search a month (1% CTR). But is that good or not? It depends. Are a handful of search phrases sending 99% of the traffic? Or is traffic distributed evenly across thousands of long-tail phrases? The Keyword reach metric helps you quantify the breadth of your presence to better manage it.

To understand keyword reach, start with the number of unique search phrases that drive natural search traffic to your site currently. What volume of traffic does each phrase drive? Your analytics package likely contains this basic data. But before we get excited, let’s apply a few filters.

First, differentiate brand phrases from non-brand phrases in your calculation. Why? Searches for your brand or company name are just table stakes. Don’t let yourself feel good about them. What you care about increasing is your share of the non-brand markets. Once you’ve separated these, determine the percentage of your traffic driven from brand vs. non-brand queries. Many retailers, for instance, find that as much as 95% of their natural search traffic comes from brand vs. just 5% from non-brand phrases. This confirms a big opportunity.

Now that you’ve distinguished between brand and non-brand traffic phrases, you can calculate your traffic per keyword yield rate on your non-brand phrases. If 5% of your search traffic (2,500 searchers) is from 1,000 non-brand phrases, your yield rate would be 2.5 searchers per keyword (2,500 searchers / 1,000 phrases). If your optimization is successful at increasing rankings in these 1,000 keyword markets, you’d expect to see the keyword yield increasing as well, perhaps doubling to 5.0 searchers per keyword.

For benchmarking purposes, you should also slice search phrase data by engine. This helps you avoid being misled by high level trends, and will show you if your optimization work is helping your Bing performance, at the expense of your Google performance, for example.

Last, make sure you can determine whether keyword reach is increasing or decreasing on a comparable basis, such as month-over-month (MoM) or year-over-year (YoY). This way you can measure the impact of your optimization work going forward in an objective manner.

Understanding keyword reach lets us augment our business case with some more powerful information: “Of our 200,000 addressable keyword markets, we are reaching only 1,000. These keyword markets drive 2.5 searchers/keyword. 95% of our search traffic comes from queries for [company name].”

Keyword placement

How highly do you rank in those 1,000 non-brand keyword markets that drive traffic? Can you do better?

We all know there are scraping tools to capture rankings. Rather than worry about ranking in markets no one searches for, and that you get no traffic for, what if you could determine where you rank for the thousands of phrases that did drive traffic? There are tools that tell you this by parsing the referral strings on engine traffic. Each major engine thankfully passes an argument in the URL which tells you what page of the SERPS (although not the actual position—yet) the searcher clicked on to land on your page. Naturally you want to filter these by engine.

Armed with this information, you can identify which phrases are ranking on Page1 (P1) of Google, Yahoo, or Bing. You can use this of course to talk up your SEO ninja skills in front of your CEO. More importantly, you can identify which phrases are ranked on P2 or P3 etc, in each engine. Perhaps you’re ranked on Google’s Page 1 in 70% of the non-brand markets. 20% of the time you’re on P2. You can begin to imagine an SEO campaign aimed at these markets, in order to get your ranking onto P1, thereby increasing CTR and traffic by many multiples.

The keyword placement metric helps you further enhance your business case: “Of our 200,000 addressable keyword markets, we are reaching only 1,000. We rank on page 1 of Google in 70% on these keyword markets, driving 2.5 searchers/keyword. 95% of our search traffic comes from queries for [company name].”

Landing page yield

Natural search is a marketing funnel: at the top are your supply of pages which get crawled. Some percentage of those get indexed. A percentage of the pages indexed get ranked. A percentage of those get clicked, etc. So an important way to baseline current performance is to map your search traffic against the pages that drive the traffic. Surprisingly, this is more difficult than it seems. In fact, it is highly likely that you do not know how many pages are on your dynamic site (don’t worry—I won’t tell).

The easiest way we find to approximate site size is to measure the number of unique pages crawled by a particular engine over a period of time. Engine crawl has slowed recently. Review your log files for unique URLs requested by Googlebot over a 60 day time frame. Or you can run a crawler over your site. Either way, you should be able to parse highly duplicated pages from the count. For our purposes, let’s assume there were 10,000 unique URLs requested by Googlebot over the past 60 days. In other words, Google thinks your site has 10,000 pages.

Naturally, you want to compare this to how many pages Google reports having indexed. If you have just 1,000 indexed, you should investigate why 90% of the available landing page “inventory” you take to market is being rejected, and then develop strategies to address the issue.

You need to understand how many of your URLs drove search traffic. Suppose 250 pages are driving your 2,500 non-brand monthly searchers. That means 250 of your 1,000 indexed pages are performing—a rate of 25% yielding 10 searchers per page. However, you have 10,000 total pages, of which 250 are performing—an actual yield rate of 2.5%. If these were your metrics, you should have many questions:

• Why are 90% of my pages not being indexed?
• Why are 75% of my indexed pages not being found at all?
• How can I identify my non-performing pages?
• How can my 25% yielding pages achieve more than 10 searchers per page?

Asking and answering these questions leads to smarter strategies and more accountable performance.

Your business case now looks like this: “Our addressable market spans 200,000 unique keyword markets and totals roughly 5,000,000 searches per month, of which we’re capturing 1% currently. The cost of acquiring that traffic through PPC totals $1,000,000 per month. Of our 200,000 addressable keyword markets, we are reaching only 1,000. We rank on Page 1 of Google’s in 70% on these keyword markets, driving 2.5 searchers/keyword. 95% of our search traffic comes from queries for [company name]. 97.5% of our pages drive no search traffic.”

Remember, you can only manage what you measure. With these metrics, you can create performance baselines and develop data-driven SEO strategies aimed at achieving specific performance outcomes. Trust in the force, Luke.

In my previous article, I shared a set of natural search metrics that help frame your natural search business opportunity and win the resources needed to improve your SEO performance. In this article, I’ll show how to use the first three metrics to begin building a kick-ass business case that your executives will find hard to resist. Before we proceed, just one word of advice from Obi-Wan, young Master Luke: Only use these metrics to sell (either internally or externally) if you have ability to deliver on what you’re selling.

Define the market opportunity

First, what’s the total size of market you’re chasing in natural search? Yeah, I see your blank stare. This number has to be your mission; it must be top of mind in order to successfully convince executives to invest in SEO. Think of this first metric as a basic marketing feasibility study. The SEO costs and tactics are irrelevant at this stage. Tell me how many people are searching for what you sell every month. What keywords do they use—besides your brand name. This exercise is different from traditional “keyword research.” We’re looking for an estimate of market demand.

To estimate keyword market demand, first look at the phrases currently driving traffic. Export (from your site analytics) all the non-brand phrases that drove natural search traffic over the last completed month. Augment this list with all your PPC phrases that outperform your average site conversion. You can further augment the list by running a spider over your site to extract onpage phrases that may be worth exploring, but this is a more advanced step and is probably counter-productive at this stage. (Trust me: your business case will be compelling enough by the time we’re done. Resist the temptation to inflate it into unrealistic fantasy data that will only hurt your credibility later.)

Historically the tools needed to conduct our feasibility study have simply been unavailable, or we were forced to make keyword research tools do a job they weren’t meant for. Thankfully, Google made its AdWords Keyword Tool, with demand data, accessible for advertisers over a year ago. Hopefully Yahoo and Bing will follow. For now, simply feed your keyword list into the AdWords tool over the comparable time frame. You can query using broad or exact match, but I’d suggest exact match to calculate more precisely how much of that market you penetrated.

A few caveats here: First, Google’s numbers are imperfect, but better than the alternatives. We find that Google will fail to report useful data for many long-tail keywords that have small demand. This is an issue since long-tail keywords are the essence of what big natural search programs are made of (another tool limitation, for the time-being). Second, we suspect the demand numbers Google reports are overstated by a factor of between 20% – 40%. So you can discount accordingly to keep a clean, consistent Google-only calculation through your business case.

At the end of this process, you want a spreadsheet with the number of keyword phrases and the demand of those markets, with a total market size that you are competing in, across natural and paid search. You have defined your market opportunity when you can confidently tell your executives something like this: “Our addressable market spans 200,000 different keyword markets, and totals roughly 5,000,000 searches per month.”

Estimate click through rate (CTR)

The next question asked will be, “But how much of that are we getting right now?” To answer this, determine what percentage of those markets your current keyword traffic represents. This is an easy and powerful calculation. You know what non-brand traffic you received from Google during the same time period (from your site analytics). Simply divide the market opportunity by actual traffic received for each keyword to determine your CTR for each keyword. You should calculate CTR on an individual keyword-level basis. For bottom-line purposes, calculate a weighted average of the individual keyword CTRs.

You have defined your CTR when you can now tell your executives something like this: “Our addressable market spans 200,000 unique keyword markets and totals roughly 5,000,000 searches per month, of which we’re currently capturing 1%.”

(Pause the movie a second. Did you catch what just happened? With just two metrics you now have the power to compare the efficacy and potential of natural search against other forms of online advertising using familiar performance metrics—without resorting to ineffective faith-based arguments or resource-begging. It gets better. Unpause…)

For added power, insert a column in your spreadsheet showing where your site ranks for each phrase (SERP page is fine for now). As you might expect, you’ll find that phrases for which you’re ranked on Page 1 of Google (here I’ll be using shorthand like P1, P2, etc.) will have a higher CTR than phrases for which you are ranked deeper. Now compare CTR on P1-ranked phrases against P2-ranked phrases, etc. Our retail clients find Google P1 consistently achieves CTR of around 10%, while P2 achieves around half the CTR (<5%). If you see similar values, the business proposition is crystal clear: There are performance gains to be had by investing in phrases ranked deeper than P1. We all know this instinctively, but now the data can do the talking for you. Note: There are tools that report on SERP rankings automatically. Or you can parse referral strings yourself on all engine traffic to capture what page of the SERPS the searcher clicked on to land on your page.

If you let it, the data can say something even more powerful. Once you quantify the performance difference between your P1 and P2 (and deeper) phrases, you can assign and model a budget value to achieving the desired lift across the desired markets—along with some probability assumptions. Don’t be surprised if that budget is much more than you’re budgeting for all your SEO efforts right now. You’ll recall my premise: your SEO is underperforming now for this very reason.

Map out traffic acquisition cost

Now for the trifecta: Knowing your addressable market size and current penetration, your executives want to know two things: How much are you spending on that performance now, and how much does “more” cost. You’re in the game. The costs of advertising via paid search continue to increase. The cost of advertising through natural search is dirt cheap. Your executives want to increase profit by acquiring more traffic (and sales) at less cost. Natural search is an obvious way to increase traffic, decrease costs and increase profit, right?

You can easily answer the “cost of more” question by exporting the average CPC value reported by Google’s AdWords Keyword Tool above. Simply add two columns to your spreadsheet, one to reflect a growth assumption on your current natural search traffic (50%, 100%, whatever), the other to estimate what it would cost to acquire that traffic using AdWords. Total it up. You’ve just created a reality-based framework for valuing natural search that executives will appreciate – even though it may total more than what you’re currently budgeting to spend on SEO.

Your message to the CFO now looks like this: “Our addressable market spans 200,000 unique keyword markets and totals roughly 5,000,000 searches per month, of which we’re capturing 1% currently. The cost of acquiring that traffic through PPC totals $1,000,000 per month.”

More can be said on this point in future installments. But notice that you’re no longer taking Quixotic approaches towards SEO. Instead you’re making a business proposition based on markets with known performance and that have real costs associated. Your data is doing the talking for you, and your CFO’s heart is warming up to you. Powerful, the force in you is, yes…

Next time I’ll discuss the remaining metrics that help measure incremental gains in performance.

What a thrill it is to kick-off Search Engine Land’s new “Analyze This” column! I applaud SEL for creating space for this industry conversation. Those of you who’ve heard me speak at industry conferences (SMX, AdTech, ACCM, SES, Shop.org) know I believe there is a strong disconnect between the opportunity natural search practitioners see (or instinctively understand) and how organizations resource and value natural search. The primary cause is that natural search practitioners have lacked a proper metrics-based business framework with which to have more powerful business conversations. Lacking this framework, the natural search opportunity fails to convince executives (internal or external) to allocate needed resources. As a consequence, natural search performance suffers, and sadly, your contribution goes under-recognized in the eyes of the organization.

I hope to help. The development of more robust search metrics has been a passion for me over the past 8 years in the field and in working with premier online brands at Netconcepts. I love measuring things. Looking for patterns. Questioning. Concluding. Experimenting. Learning. Then doing it all over (and developing products that automate those processes). I very much view natural search as a form of direct marketing. One my professional aspirations is to help elevate natural search from that of “black art” to a genuine business discipline capable of producing sustainable results more predictably (OK, I have ambitious tendencies). And as my company and colleagues would attest, I am a firm believer in the old management truism “you can only successfully manage what you have successfully measured.”

So what I want to offer you in my regular contribution to this column is simply this: How to systematically improve your natural search performance. More specifically, how metrics can help you win the resources needed to grow your channel; how to calculate them; which metrics matter; how to shape them into powerful business conversations; how to evaluate the smorgasbord of SEO tactics available to you, and their effectiveness at driving performance for you. When your CEO knocks on your door demanding you double your search performance in 6 months, I want you to know how to assess the feasibility and requirements of achieving that objective—or at the very least, how to tell your CEO what is more attainable and why.

Let’s get started.

The natural search opportunity

Allow me to humor you with some industry facts:

• Search, as an industry, is growing from a $12B industry in 2007 to a $25B industry in 2011—100% growth in just 4 years (SEMPO)
• CPC costs are on the rise—Google’s average CPC rose 15% last year
• Consumers click on natural listings 85% of the time according to research (from Forrester)
• Meanwhile marketers spend 85% of their search budgets on paid search (according to Marketing Sherpa)
• 56% of Google queries have 0 paid ads placed (each of those queries present 10 “non-paid” or natural search results (or for the forward thinking, “ads”) on the results page) (according to comScore)

Clearly, natural search presents marketers with a much bigger pool of potential consumers—each of which can be acquired much more profitably than by any other means.

But as practitioners, how do we normally communicate this opportunity? Most search geeks (I mean that lovingly and inclusively) default to tactic talk and irrelevant metrics: begging for an hour of IT or copywriting or mechanizing to modify HTML elements on templates, H1 tags, page titles, or add links, or rewrite a URL or two. Then you get shot down because when pressed for “what’s it worth,” you start fumbling for something impressive to say (”um… thousands! Maybe millions” are gems I’ve heard over the years). And when that fails you resort to “leap-of-faith” arguments that CFO’s just don’t buy – or at least not for very much money. The standard SEO metrics you do share have no connection with business—you know what I mean. Things like PageRank flow, inbound links, keyword density, engine distribution, ego term rankings.

And that’s the nub of our issue. Winning the Darwinian resource struggle requires not that we lose our passion or faith in search, but that we realize it; and I am saying that we realize it by first talking business metrics before we talk implementation tactics.

Show me the money

Ultimately if we want resources allocated towards SEO work, we must be able to communicate the value of that work in return on investment (ROI) or return on ad spend (ROAS) language. That seems simple enough. Yet the reality of measuring natural search is complicated by the long-tail and distributed nature of natural search.

How do you define what is actually incremental traffic or sales? One month you get three visits for a long tail phrase. Next month you get just one. Did market demand decrease? Or did your rankings decrease? Or did both stay constant while other SERP page competition stole your traffic? And practically speaking, how can you possibly care about such small volumes? You don’t worry about the height or width of each blade of grass in your lawn (I hope). Yet successful search channels are comprised of just that: hundreds of thousands (or millions) of non-brand phrases each acquiring handfuls of searchers every month for various pages of your website.

In addition to the scale issue, many factors impact natural search performance, and many stakeholders are likely influencing what the engines are seeing and responding to. Can you correlate gains or losses to specific site changes? Or is your performance driven by an invisible hand of unnoticeable activities—perhaps the combined effort of user interface enhancements and keyword-rich copywriting tempered by new AJAX navigation schemes, all of which may have been deployed at various times over the past quarter or year. It’s complicated!

So our challenge is this: not only must we speak in ROI/ROAS business language, but we must also understand what’s driving performance currently, what the level of effort put forth has been, and communicate how we expect actions to contribute towards that positive ROI/ROAS we see.

That said, here is a framework of progressive metrics we have developed that take many of these factors into account in order to drive a stronger natural search business conversation:

• Market opportunity
• Clickthrough rate
• Traffic acquisition cost
• Keyword coverage
• Non-brand reach
• Landing page placement
• Landing page yield
• Incremental traffic / revenue
• Return on investment / return on ad spend

In my next few installments, I will walk through each metric, how we derive them, their relationship to each other, and how to use them in a practical manner.

In recent years, many retailers have implemented powerful attribute-based (a.k.a. faceted) navigation systems that make it easier for customers to filter, sort, navigate, and buy from a retailer’s inventory. Most of Netconcepts’ clientele who have implemented these technologies report good results, increased conversion and sales. From the natural search perspective, many such marketers are pleasantly surprised to find increases in the number of their pages indexed by Google upon implementing such systems. But in this case, what’s good for users, is actually not so good for search engines — or for your bottom line.

Recently, Google Webmaster Central took a position on this by pronouncing the “infinite filtering” and “resulting page duplication” produced by such guided navigation systems as negative for bots, and urged that the “blech” of duplicated pages be cleaned up, by saying:

“Another common scenario is websites which provide for filtering a set of search results in many ways. A shopping site might allow for finding clothing items by filtering on category, price, color, brand, style, etc. The number of possible combinations of filters can grow exponentially. This can produce thousands of URLs, all finding some subset of the items sold. This may be convenient for your users, but is not so helpful for the Googlebot, which just wants to find everything – once!

….One fix is to eliminate whole categories of dynamically generated links using your robots.txt file… Another option is to block those problematic links with a “nofollow” link attribute.”

While useful for consumers, the attribute filters also produce unique URL parameters based on elements like product colors, sizes, price, popularity, the number of results per page, and the pagination construct itself, resulting in dozens or hundreds of permutations of unique crawlable URLs. But these unique URLs contain content that is completely similar and duplicated in nearly all respects. And duplicate content of this magnitude hurts your natural search marketing performance.

Here’s an example from the Artful Home brand: Compare this page to this page. Unique URL addresses, but same content. Click on the “price range” or “color” or “medium” attributes in the left navigation and you’ll get even more URLs with the same content. In fact Google has indexed 20 URL permutations containing the exact same content for this one page. Same title tags, headings, body copy, and more.

Below are five natural search marketing performance issues introduced by attribute-based retail navigation models (as typically implemented):

1. You end up creating self-competition between your site’s own pages. (If you were Googlebot, which of these do you consider the authority page?)
2. Your pages don’t resonate with searchers. (People seldom include attributes like price in their search queries)
3. You bloat the engine index. (You don’t have as many pages indexed as you’re bragging about to your CEO.)
4. You burn your “crawl equity.” (More of your unimportant pages get crawled with each visit a bot makes.)
5. You fragment your available link popularity (PageRank) between so many different versions of the same/similar pages.

To deal with the situation, merchants with attribute-based navigation systems need more sophisticated strategies and execution capabilities.

For example, the on-page attributes that create the navigation scheme should be researched during the design phase to ensure your resulting landing page themes are consistent with searcher vocabulary. Filtering options that are not beneficial to your natural search performance should be similarly evaluated, and tactics put in place that make any “false” landing pages uncrawlable for bots. A combination of nofollows, meta noindexing and disallows strategies should be employed for this. This is easier said than done as most merchants require IT resources to modify the site architecture in this way.

(If you’re in this boat and expending any serious amount of IT effort is not an option, you may wish to consider a proxy site technology such as GravityStream to generate and apply such rules-based changes simply and automatically across your large scale site. (Disclaimer: I manage the GravityStream product at Netconcepts.) One benefit of this approach is that you can, without involving your IT department, automatically resolve the many parameter-filled URL permutations created by such navigation systems into singular, authoritative versions of each category, subcategory, and product level URL — a form of “canonicalization” via intelligent redirection. By eliminating page duplication in this fashion, you maximize the distribution of your available link popularity to the greatest number of unique landing pages.)

Google’s position on this matter may seem to contradict their Webmaster Quality Guideline around “making your site for users not search engines.” After all, attribute-based navigation systems do offer many great user benefits. But they will unquestionably dilute your natural search marketing performance if not engineered with search engines in mind. Today’s savvy merchant is already implementing sophisticated strategies to capture the best of both worlds.