Golden Tablets

I’m not talking about Joseph Smith’s discovery, I’m talking about Apple’s. The traffic from iPads is more valuable than traffic from desktops for many in the eCommerce sector.

Our data suggests the traffic from tablets as a class to be worth ~15% more than traffic from desktops. Traffic from the Kindle Fire seems to be less valuable, but that makes up a pretty small fraction of the total, therefore doesn’t degrade the overall value of the traffic significantly.

Different Demographics

Quite likely, the differences above are in part demographic. Folks with iPads are early adopters of technology with significantly higher than average household incomes.

The disparity with Kindle Fire may represent how ads are displayed on a generally smaller screen, but also, perhaps a function of the price point attracting a less affluent user.

Different Usage Pattern

People use tablets differently than desktop computers. I have heard from folks in the know that something like 80% of tablet traffic takes place between 6PM and 9PM in the tablet’s time zone.

More than 95% of tablet traffic happens over a wireless signal rather than a cell tower connection. Tablets are not, by and large, mobile devices. The tablet should be seen as a couch companion. People mostly use their online functionality while watching TV.

More Valuable & Less Expensive

Not only is the traffic on tablets more valuable than average, it is also less expensive by volume than desktop search. That is to say: the same bids on tablets put you “higher” on the page than they do on desktop. This is a function of the fact that few search managers are separating campaigns by device at all, and many of the ones who do have left tablets bucketed with smartphones.

In either case, the blended average traffic value is leaving opportunity on the table, and splitting tablets into a single “mobile” bucket with phones may be worse than doing nothing at all. Grouping tablet traffic with desktops may be an acceptable compromise for busy SEMs.

These facts should send online marketing directors for major brands scrambling to figure out whether and how tablets are being split out from traffic from other channels.

Also, those brand advertisers who spend significant money on television ads might consider whether the look and feel of the normal ad landing pages is consistent with the messaging in the commercials. Might there be an opportunity for a separate set of landing pages tied to the themes of the commercials running at the moment?

The Smartphone Conundrum

Your mileage may vary a great deal, but for many advertisers, particularly those with a wide range of products and services to offer, the online conversion rate for smartphones is a small fraction of that of desktop devices.

We have heard that this will all change with better, easier online payment types, and that NFC will make our phones the offline shopping credit card of choice. Perhaps. It’s also quite likely that it’s really hard to sort through lots and lots of different items on a tiny screen. That problem won’t be resolved until human eyesight improves and fingers get pointier.

We know that something like one third of all smartphone searches carry local intent, that is: the user is trying to find a physical location. For businesses that are online only these users are the wrong group on which to spend advertising dollars.

To advertisers with a wide brick and mortar footprint the online conversion metrics may not tell the whole story. Perhaps these folks are converting offline instead and the value of the traffic is materially understated?

It is worth testing with POS coupons to get a sense of this, but in the meantime, you might at the least restrict the degree to which that online to offline assumption is being spent within a reasonable geographic proximity to your bricks. Spending beyond online ROI metrics on smartphone traffic 100 miles from the nearest physical location is likely unwise.

Context Is Coming

If marketing hasn’t become sufficiently complex yet, might we think differently about our bids and even ad creative to smartphone users whose device is motionless vs moving 2 mph vs moving 30 mph vs moving 60 mph?

We must crawl before we walk and run before we fly, but SEM isn’t getting any easier any time soon.

None of these refinements will make much difference unless and until the mobile experience we offer our users improves. The average quality of mobile user experiences is low, and it is likely that there are heavy penalties coming soon for brands with poorly optimized mobile user experiences. There are rumors floating around that the penalty for high bounce-rates for organic listings in mobile and mobile QS in ads is about to be increased.

Conclusion

The big picture of all this is simply that we must not expect all users to be equally valuable to our business regardless of the device they’re using and the context in which they’re using it.

We need to market to them smarter, now, and build better experiences for all users in the near future. The growth rate of searches from devices other than desktops and laptops suggests that the penalties for ignoring these differences will only get bigger as the year progresses.

The earliest adopters of sophisticated paid search practices were those companies with deep roots in direct response marketing.

Roots in Direct Response Marketing

In e-commerce, it was the traditional catalog companies that figured out search first. The cost of a book (catalogs are called “books” by insiders) in the mail looked a lot like the cost of a click, and conversion rates looked a great deal like catalog response rates.

Catalogers were steeped in notions of segmenting data by source, by geography, by season, by interaction history, as well as creative and offer testing. They were also comfortable driving by that information.

In finance, it was the credit card companies who were quickest to understand the connection to direct mail and applied the same intellectual rigor that served them in those pursuits. Internet pure plays that grew up in tracked marketing were quick adopters as well.

Many well-established companies that relied historically on TV, newspaper circulars, radio and other media buys that are inherently less trackable were generally slower to adapt to advanced SEM practices, because driving marketing decisions by the numbers wasn’t deeply ingrained in their culture.

(Note: I’m casting no aspersions on the brands below as I don’t know how they’ve driven their programs, just giving samples of the types of companies I’m talking about.)

Roots in Media Buying Not DR

Big Box retailers, banks, clothiers, real estate brokerages and others often still make the mistake of thinking of paid search the same way they think about print ads: impressions, audiences, context and creative.

We often find horrendously broken programs, spending money with no rhyme or reason as to how the money is spent. Mismanagement of the critical non-brand spend is often hidden by brand advertising performance.

In many, many of these cases the objective may be brand building. In other cases like those below, building brand awareness is the only realistic objective.

Purely Brand Building

Where sales are widely distributed and untrackable (Budweiser, Levi Strauss, Ford, Blistex, Paramount, the Florida tourist bureau), or not the objective at all (a presidential campaign, Exxon, BASF), branding is the right way to advertise.

But if brand advertising is the objective, there is no reason the same paid search rigor won’t prove useful. Simply spending a budget, targeted at a blanket Cost per thousand impression (“CPM”) or targeted to capture a certain “impression-share” is not the way to get the most bang for the buck.

How Should Value Be Determined For A Branded Campaign?

For a brand building program, careful consideration should be given to how value is determined. I’ve heard people compare the CPMs available in search or display to print or TV and thought: huh? Talk about comparing apples to oranges!!!

The cost of the impressions may be similar, but what about the value of the impression to the brand? Certainly watching a 30 second TV spot has a different impact on a person than does a text ad that appears on a webpage along with 20 other text blocks.

The human user searching for something relevant to your brand is far more valuable than the regionally/demographically targeted TV viewer (who may be getting a snack during your spot), but the impression itself conveys much less about your brand.

Paid search is managed well when the cost of advertising is married to the value of the advertising. We must first consider carefully the value associated with different types of brand interactions.

A paid search ad may have a very well understood and easily measured cost, let’s say $50 for a day, but what you get for that cost may be a complex set of brand interactions that look like this:

How do we make sense of that value?

The best approach is to create some sort of common value metric hopefully based on data, but minimally based on good intuitive reasoning. It might look like this:

Now, this is a bit idealized, but you get the notion. In this case, the value appears to be worth the costs and that ad can bear a CPC of $0.63. Text ads with similar numbers of impressions may have dramatically more or less value based on the quality of interaction after the exposure. Evaluating by CPM is the wrong way to think about this.

The complexities don’t stop there. For State Farm, Century 21, or Talbots, shouldn’t the brand advertising be more valuable in close proximity to the brick and mortar presence, where dropping by the store or office may become more top of mind and more likely?

Shouldn’t we place even more branding value on traffic coming from mobile devices near the physical locations? Maybe smart phones > tablets > desktop?

If there is more valuable brand advertising available than you’ve budgeted for, how should we prioritize? We may find that we can spend the whole branding budget on very tightly aligned keywords.

Is it better in this case to spend the whole budget on the highly targeted tail, or on a handful of head terms that are less well targeted to your brand?

If you’re the Romney campaign, for example – which type of keyword would you spend the most money on: “Obamacare”, “Health Insurance Reform Act”, or “Health care policy debate”?

During the primary, you may want to appeal to the base more than others and focus on people searching for Obamacare. During the presidential campaign, maybe you don’t need to advertise as much to them, but really want to hammer the “Health care policy debate” searchers on Google, Bing, and YouTube.

The point is simply this: driving brand advertising dollars ‘by the numbers’ pays dividends just as it does with direct response. The complexities of brand value measurement make applying this rigor more challenging, but every bit as worthwhile.

It is not at all uncommon for senior management to become unhappy with a terrific and progressing paid search program because of the way we communicate results.

Averages and aggregates lie, as I’ve argued many times, yet they are also an essential tool of managing a paid search program. Even the manager in the weeds cannot look at the most granular performance data and make any sense of it.

We have to look at data aggregated over time, by category, by subcategory, by match type, by geography, manufacturer, the list goes on and on. To really understand what’s going on, we have to study the data six ways to Sunday to get the complete picture, but that can create problems for reporting out results to senior management.

Management doesn’t have time to study the details and aggregated reports can make great performance look mediocre or worse.

Four Reasons Management Can Be Unhappy With A Great Program

1.  Judging the program by metrics outside of the paid search manager’s control.

The most prevalent example comes from blending brand/navigational search in with competitive non-brand search. Since brand search is almost entirely driven by other marketing efforts, the search manager should neither receive credit nor blame for those numbers. I detailed this one a couple months ago, so I’ll leave it at that.

2.  Management sets goals that aren’t grounded in reality.

Any good SEM firm can point to plenty of examples of putting up gaudy growth numbers when taking over poorly managed programs. But there are also instances when new senior managers come in with “ambitious” goals for a program that is already hitting on all cylinders.

“We’re planning on 100% YOY growth this year, so you’re going to have to work really hard!” Failure to understand market constraints can make excellent performance look “way below our targets.”

3.  Looking at the wrong numbers on the report.

Less common now than in the past, but still not uncommon: people who don’t get search will often latch onto metrics that are indicative of nothing. For example:

• What’s happened to our CTR?!?
• Why is the conversion rate down?
• Our average CPC has gone up! That’s terrible! What’s going on!
• Our average position dropped, someone’s head must roll!
• How many keywords have you added this week? Get cracking!

All of these can be useful diagnostic markers for identifying potential problems and opportunities, but many folks still mistake these for KPIs. They aren’t, and certainly aren’t for management level. A paid search manager might add a bunch of ads that perform differently in the marketplace than others and still be a tremendously helpful addition.

Consider the following example, where the non-brand search results typically look like this in aggregate:

The intrepid search analyst finds a treasure trove of great new keywords and launches them, producing the following results:

Uninformed upper management goes nuts:

“Haaaaaaaaaaarumph! Click-through rate is down 50%! Conversion rates have dropped 4%! CPCs are up nearly 50%! Heads must roll!”

Focusing on the more meaningful available metrics would yield a different response. “Wow! Sales are up over 130% and the ROI actually improved! That’s fantastic!”

4.  Effectively hiding the most meaningful numbers from management.

This manifests itself in many different ways. Let’s consider the case of a major auto insurance company. They take leads in exchange for rate quotes. The average lead is worth $250 to them.

The outstanding search marketing manager conducts research on the back end performance of leads and learns that the leads from certain types of keywords (or geographies or whatever) are significantly more valuable than others (say $500), and that leads generated from searches for “cheap/discount/low-budget insurance” are worth considerably less (say $150).

Looking at the past data this way reveals that they’ve been overspending in some areas and underspending in others. So, she adjusts the targets to generate better ROI for the company.

But senior management wants her fired because from their vantage point, the numbers went south big time.

“Spend is up, leads are down, and the CPL is way out of line with our target!”

Understanding the actual lead values, it’s clear that in fact the search marketer has made the right moves for the company, but if management doesn’t see those numbers…

This phenomena can take many different shapes and forms. In ecommerce, different product lines have different margin structures making a $100 purchase in a high margin category much more valuable to the firm than a $100 purchase in a low margin category.

If the actual margin data can’t be fed into the system dynamically setting different efficiency targets by categories is the next best solution, but that leads to difficulty interpreting the numbers when aggregated across categories.

In this example, the first week’s data reveals fine results, in line with efficiency metrics necessary to maximize operating margin from each category.

The next week sees a seasonal spike in interest in the high margin products and a decline in interest in the low margin items. The well managed program hits the category efficiency metrics again, maximizing operating margin for the company, but the aggregate results showing only costs and sales seems to indicate a decline.

It’s important to note here that from the aggregate numbers there is no way to tell whether the program has gone downhill or continues to perform well. The same aggregate numbers could in fact reflect poor management and missed opportunities.

This is not the fault of upper management’s misunderstanding as much as the difficulty inherent in communicating the complex truth, which is particularly challenging when we can’t present the numbers to which we’re actually driving (net margin in this case). Many companies can feed margin data into a good bid management platform either at check out or in a back feed, but some companies simply can’t tie that thread.

As programs become more sophisticated — incorporating lifetime value considerations, what keywords generate more new customers than old, more business customers than consumers, etc. — the communication and evaluation metrics become that much more challenging to convey

Working Towards A Better Solution

There is no perfect solution here, but there are a few approaches that can help:

• Getting everyone in senior management to have a basic understanding of what the channel can and cannot do, what controls we have available, what the goals of the program are, and the challenges of viewing aggregated data is a great start. The more those folks understand about how search works the more they’ll appreciate the good work you do. Keeping management ignorant can help the incompetent manager, but creates jeopardy for the supremely competent.
• Sharing more detailed reports to management that better reflect all the nuances. This is a great solution if the management team will go along with and actually review more complex reports. The challenge is many managers operate under the dashboard principle: “I just want to see either green numbers or red numbers. Don’t tell me it’s more complicated than that!”
• Determining and reporting out an “all-in value” metric that is consistent with the program’s management goals. If the goal of the program is to drive gross margin dollars, then the aggregated reports should show gross margin dollars, not just sales. If the actual dollars can’t be determined establishing a best proxy can make sense. If experience and research suggests that the advertiser gets different types of leads which have inherently different value,

For example: homeowner’s insurance vs car insurance vs personal property insurance vs motorcycle insurance, and that within those leads of differing value there are classes of keywords within each product that drive leads of different value (homeowners insurance from Beverly Hills vs Tumbleweed, Texas), then assigning and reporting out some approximate dollar value tied to each lead collected will give a much truer picture of the health of the program week-to-week than will aggregate lead counts that treat them all equally.

This notion can be extended, to include a dollar value associated with new customers vs existing, different customer types, non-conversion success metrics like email sign-ups, etc. Assigning assumed $ values does carry the potential pitfall that the folks in accounting look at the report and flip-out because the dollars reported aren’t “real” in their minds.

We shouldn’t avoid complexities because of the communication challenges they present. Raising the bar demands a nuanced approach, and it is incumbent on both the paid search managers and senior management to make sure the lines of communication and the reward structures are aligned to encourage raising the bar, not lowering it.

Werner von Heisenberg courtesy of Wikipedia

In 1927, Werner von Heisenberg documented what he referred to as an “Uncertainty Principle” governing quantum mechanics.

The Uncertainty Principle holds that an observation cannot precisely reveal both the position of a particle at a point in time and its momentum. The more the observation reveals about one, the less the observer can know about the other. A similar principle governs paid search.

Physics sidebar: feel free to skip! If memory serves, the notion is that the act of observation impacts the object. To observe anything — by sound echo-location, sight, touch — we have to bounce something off of the object we observe.

For big things, this is irrelevant. Shining a flashlight on a tree doesn’t impact the tree. However, in the world of subatomic particles, bouncing photons or anything else off a tiny particle has a big impact.

If you want to know where a particle is at an instant in time, you have to “hit” it hard to get the answer quickly, but in doing so you impart a huge and unpredictable change in momentum to what you’re trying to observe.

The Paid Search Uncertainty Principle

This reads: “The amount of variance tolerated in Advertising Efficiency (“E”) times the amount of variance tolerated in the volume of advertising spend (“V”) is greater than some constant, K.” As the variance in one approaches zero, the variance in the other approaches infinity.

Okay, okay, enough of the physics metaphor, the idea is this: you can’t control both spend levels and efficiency metrics. The more you predetermine one, the less control you have over the other.

This fundamental law of paid search may be its least understood, as many companies determine an efficiency target and then simultaneously fix a rigid budget for how much they will spend in media. Those companies are often frustrated with their results.

Fixing a budget for ad spend translates to the following:

“I’m going to spend this amount of money regardless of market opportunities. I will flush money down the toilet if I must in order to spend my entire budget. If, on the flip side, the market conditions are such that I can make $20 in profit for every $10 I invest, I will nevertheless stop spending money when I reach my budget.”

Fixing an efficiency target translates to the following:

“I am going to spend my advertising dollars at an efficiency that makes sense for my business. I will spend an unlimited amount of money if I can do so efficiently. I will spend no money if market conditions dictate that I can’t spend efficiently.”

Stated as such, the Uncertainty Principle clearly follows.

Yet enterprise-level advertisers try to violate this principle all the time:

“We’re going to spend $1 Million this month, and we need to see an ROI of 5 to 1 (or a CPL of $30).”

It’s nice to have goals, and it’s smart to try to forecast what you’re likely to see, but at the end of the day market conditions may dictate that you pick one or the other target to hit, or you’ll likely end up missing both.

Important caveat: this assumes that your program is well managed. We’ve taken over many horribly mismanaged programs and both grown the spend and greatly improved the ROI.

But all things being equal, for any well managed program there is a trade off between efficiency and volume that has to be understood.

Paid search managers have control over many pieces of the game:

• The user searches that will and will not fire an ad
• The text of the ad fired by a particular search
• Where ads are served (both domains to a greater or lesser degree, and geographies)
• On what devices they are served
• The maximum amount the advertiser pays for a click on a given ad under almost any circumstance

As I’ve argued relentlessly in the past, we’d like more and better controls over many of the above, but we certainly do have a large degree of control.

But it is also important to recognize those factors paid search managers do not control:

• The volume of user search in a category. If the users don’t search, we can’t serve ads.
• How much other advertisers are willing to pay for clicks. If other advertisers behave irrationally, or change their bidding practices significantly, it will have a material impact on the marketplace opportunities.
• The advertiser’s selection, pricing, content quality, and promotions are often outside of the control of the paid search manager. These impact the value that can be extracted from traffic and thus the amount the paid search manager can spend for that traffic.
• Competitors’ promotions, selection, content quality, pricing, etc. Significant changes in the business practices of competitors impacts CTR, QS, and conversion rates for their ads and thereby shifts the Volume v Efficiency landscape in ways we can’t control.

The point of this isn’t to make excuses. Indeed, if the goal of enterprise search advertising  is to spend a large budget, there is no excuse for spending significantly more or less than the budget. If the goal is to hit an efficiency metric (CPL, ROI, Margin to cost ratio, whatever) for non-brand search, there is no excuse for missing that target.

However, introducing two targets often creates an unsolvable problem for paid search managers of any size.

Hitting both an efficiency metric and a volume goal requires good execution, yes, but it also requires favorable market conditions over which the paid search manager has no control.

Paid search managers must be held accountable for those elements that are within their domain, but holding folks accountable for conditions outside of their control is both unfair and unwise.

Advertising on your brand allows you to:

• Control the message. The text of the message can be kept fresh, highlighting promotions, shipping cut-off dates, whatever makes sense for the brand.
• Direct traffic. Site-links provide an opportunity for users to navigate to the next page deeper rather than just the home page, thereby aiding conversions.
• Occupy real estate. Taking up more space on the SERP means less leakage to affiliates and competitors.
• Capture incremental traffic. In some cases, our research shows that paid search ads on brand names do indeed bring in some incremental traffic. Your mileage will vary, so test this yourself, but in some instances the advertising more than pays for itself without any other considerations.

However, the fact that it is generally wise to advertise on one’s trademark doesn’t change another crucial principal of paid search marketing:

Never, never, never mix the results of brand advertising with competitive non-brand advertising.

All averages lie. Blending the results of brand and non-brand search doesn’t just lie, it will kick you in the teeth and take your lunch money, too.

Mixing the results of brand and non-brand paid search results leads to a false sense of scale and efficiency for the program which in turn creates chronic problems down the line.

The greater the fraction of overall “paid search” sales brand ads represent the greater these problems become.

Problem 1: Lack of Control

The core problem is that the paid search manager has very little control over the volume of traffic, the conversion rates, or even the cost of brand advertising. The paid search manager can and should test and adjust copy for maximum positive effect.

He or she can and should make sure site links and seller ratings are used to full advantage. S/he can and should test landing page versions and messaging to wring the most successful visits out of brand search traffic.

Assuming the good paid search manager has done these basics, they’ve exercised about all the control they have over brand search performance.

Other than guarding against some crazy CPC penalties Sid Shah and we have seen, the ads will serve at the top of the page for most folks at low cpcs and there won’t be any leverage to get more.

Traffic and conversion volume on brand search will rise and fall largely, if not entirely, as a result of offline marketing, brand awareness, friend referrals, and loyal customers navigating to your doorstep; the paid search manager controls none of those factors.

We’ve seen brand sales account for anywhere between almost nothing for internet pure-plays in start up mode, to over 90% of overall sales for very well established brands with huge offline advertising budgets.

Evaluating the performance of a paid search program by looking at overall numbers, when 70 – 90% of the conversions are on your brand, is ludicrous. You end up being praised or scolded based on factors entirely outside of your control.

Competent people want to be evaluated based on what they control, and in search that’s the performance of competitive non-brand keywords.

Problem 2: Disconnect Between Average Efficiency & Incremental efficiency

We’ve long preached the importance of understanding more than just the average ROI of your paid search efforts, but also the incremental efficiency of the last dollar spent.

Predicated on the notion of Diminishing Marginal Returns, smart marketers will always get more ROI from the first dollar of advertising than the last dollar of advertising, because s/he always picks the lowest hanging fruit available.

Understanding the incremental ROI will let you predict what the next chunk of ad spend is likely to generate.

Because of Diminishing Marginal Returns the incremental ROI is always worse than the average ROI — if it isn’t you’re doing something wrong. Our studies of Bid Simulator data find that within competitive non-brand paid search the incremental ROI is usually on the order of 60% – 80% of the average ROI.

Your average non-brand ROI may be 5 to 1, but additional spend over that same period is likely to be at 4 to 1 or 3 to 1, which may be under water for your business.

However, that disconnect pales in comparison to the disconnect between the two when the average includes brand sales.

We see instances where an advertiser sees an “average ROI” (including brand sales) of 8 to 1, where the competitive non-brand average ROI is 1 to 1, and the incremental ROI is significantly worse than that.

The brand may say: “We’re comfortable letting the overall ROI drop to 7 to 1 for now” but oftentimes they don’t seem to realize that they’re feeding money into a shredder at that point.

Let’s think about a 1 to 1 ROI. You spend a dollar in advertising to drive a dollar in sales. It would be just as efficient to have your paid search manager place orders on your website using the marketing budget to pay for it…and then throw the merchandise in the trash can when it arrives!!!

The 1 to 1 ROI may make sense for some folks, and that’s cool. The point isn’t that you shouldn’t spend for branding purposes, just that incorporating brand data hides the likely ROI on the next dollar spent.

Problem 3: Complete Misread Of Online To Offline Spillover

Hugely misleading studies, like this one, suggesting that for every dollar generated online by search ads there are $6, $7, I’ve even seen a claim of $10 in sales generated offline, make people believe that they’re spending money cost effectively when they aren’t. Not one of these studies has made the brand non-brand distinction.

Obviously, plenty of people want to find a store locator, the nearest State Farm Agent, business hours, in-store availability of what they’re looking for, etc. before they hop in their car and drive to the store or business.

They search for the brand they’re about to drive to, and you can bet your last dollar that the ad that shows up after they search for you by name isn’t capturing much incremental traffic.

Problem 4: Pulling The Wrong Levers

Because the misunderstanding of advertising efficiency is SO profound companies compound their problems by taking money out of other offline channels to pour more money into search because it seems so much more efficient.

“We’re getting an 8 to 1 ROI from paid search and only 2 to 1 from catalog prospecting; let’s dump the catalog and put the money into search.”

I’ve seen this happen more than once with budgets for catalogs, infomercials, circulars, TV ads, etc slashed to put more into search.

Sometimes that’s the right call, but often it’s based on the fundamental misunderstanding we’re describing and it leads to a death spiral.

The advertiser presents us with more money to spend in search and says: “we need sales to grow by 50% to cover the loss of the other marketing efforts”, and frequently paid search sales go nowhere or even decline because the advertising that was driving the brand search was cut off.

Problem 5: Inevitable Disappointment

As a result of the first four problems, the advertiser will invariably end up disappointed with their paid search manager or agency at some point.

Frustration comes because incremental advertising spend in search is hugely inefficient and this comes as a surprise for some reason.

Or, overall sales goals aren’t met because brand sales sag unexpectedly and the paid search manager has no way to compensate. Or the reckless spending encouraged by the combined view is a symptom of deeper business problems resulting in a trip through Chapter 11.

Or someone else in the organization — a CFO, a CEO, a new search manager — comes in, pulls apart the numbers and is horrified at the wasteful spending in competitive non-brand search that was previously hidden from view.

We’ve seen all of these scenarios play out, despite the fact that we have always encouraged our clients to evaluate performance strictly by the non-brand numbers.

With few exceptions, when goals are expressed in terms of aggregate brand and non-brand performance frustration is inevitable.

Advertisers end up frustrated with the paid search manager for not controlling that which s/he doesn’t control, and the paid search manager with their bosses for not understanding this fundamental disconnect.

The Charlatans Don’t Help

Unfortunately, many agencies, engines and others who feed on online advertising have a vested interest in blurring the brand, non-brand distinction.

These folks exaggerate the complexity of search funnels, buying cycles, and now attribution across channels and encourage advertisers to look at data more ‘holistically’ and with less granularity. That’s garbage!

Advertisers should look at data more intelligently, not throw up their hands and call it holistic marketing.

Intelligent use of data includes attribution of credit within paid search and across channels — as we do — and may include complex media mix studies to gauge optimal advertising spending levels for each program.

There are many valuable ways to look at aggregated data. Performance by category, subcategory, geography, keyword specificity, landing pages, number of ‘tokens’, and other useful classifications can reveal actionable insights.

However, folding in brand results never aids clarity and only causes confusion. While it may, in the short run, make paid search look better than it is, it will invariably make the paid search manager look bad in the long run.

Overall, the discussion was great, but chopping up the issue into facets missed the broader implications of PSE. So, I thought I’d close out my blogging for 2011 with a prescription of my own and how such changes could impact paid advertising.

Let’s start from first principles and address the question: “What do users want from a search engine?”

The most concise answer might be: we want the engine to provide results that match our intent.

When I search for “pictures of Abraham Lincoln” I want the results to be images of Abraham Lincoln, not websites that have those images. If I search for “Newton’s gravitational constant” I’d like PSE to give me the number, not websites where I might find that information. If I search for Walmart, why not take me to their website directly, or perhaps to a map if I’m searching on a mobile device?

But herein lies the rub: sometimes the user’s intent is obvious, sometimes, as with that last example, it’s somewhat unclear, and other times it is utterly ambiguous. Google and Bing try to guess based on the behavior of other users who conducted similar searches, based on the browser’s past activity, based on geography, and a host of other factors.

The engines have done an amazing job of “organizing the world’s information” as the Google folks describe it, and the intent matching continues to improve, but we’re still pretty far from understanding exactly what Susy wants this time when she searches for “Golf”.

The perfect search engine will not be able to read user’s minds either – at least, not in my lifetime – but, until we get to clairvoyance, the next best notion might be: how quickly can PSE return results that match the user’s real intent? In many circumstances, the fastest way to get comprehensive results is to ask follow up questions.

The perfect search engine should recognize degrees of ambiguity and respond with:

• Exactly what I ask for when the intent is clear,
• A range of potential options (universal search) when it’s less clear, and
• Appropriate follow up questions when the answers will get the user what they want quicker than they will get it from an array of widely disparate choices.

Many people search for “furniture” but very few actually want something that vague.

Quick refinement options, targeted appropriately might be the best response. For example: “Indoor or outdoor furniture?” “What type of furniture?”

Most good websites do this already, through navigation drop downs and shopping widgets.

The results of these refined searches would get users much closer to their end goal and save them lots of time looking through websites that don’t carry what they’re actually after.

How The Perfect Search Engine Would Affect Advertisers

Advertisers would benefit tremendously from this as well. Instead of competing for traffic they may not actually want (because the user really wanted a mission-style dining room set, and they sell inexpensive living room and bedroom furniture) they can compete for traffic after the user has clarified their intent. This would improve conversion rates, and save users time as well.

Let’s think about “flights to Cancun”. Spitting out websites that will then ask questions about dates and accoutrements is okay, but the future might see the engines take all the information needed from the user and then show the choices from the airlines directly bypassing the OTAs entirely.

In eCommerce, perhaps the PSE would provide the same type of navigation options to get user’s to the product level before shipping them off to the advertiser’s site.

PSE has to figure out how to present the right questions and options, and exercise judgment in when to seek follow up and how many refinement questions to offer.

Clearly, we don’t want PSE to turn into Mr. Clippy.

Effectively PSE could evolve into the uber-website, taking most of the weight off of internal site navigation because the users come into the advertiser’s site at the right level of depth in the first place.

Perhaps the perfect SERP would have a handful of “best guesses” at the top — paid or organic or some of each — with a prominent set of pull down refinements targeted to the category.

User: “Attorney” PSE: “Here are some general listings for “Attorney”, but to expedite your search, please select the type of attorney do you need from the following list.”

Perhaps today’s engines are heading down this path already. Quite possible that they’ve been down this path and found it to be less user friendly than I imagine (not PSERP but LSERP where the “L” is for lousy).

Perhaps this isn’t as different from “instant” as I think it could be.

What Does The Perfect Search Engine Look Like For Advertisers?

The other question the panel mentioned but didn’t have time to flesh out was: is there a difference between what users would find to be PSE and what the advertisers would find to be PSE?

I think the answer is “no”. Satisfying the user’s intent on the first non-engine page will save users time and frustration, encouraging more search. Getting users what they want on the first click will greatly improve conversion rates for advertisers by better qualifying the traffic.

Increasing the value of the traffic will increase the price advertisers are willing to pay which will more than compensate PSE for the reduction in fruitless clicks. Win-Win-Win.

A New Year’s present for all of us?

There are plenty of good reasons to do so regardless of the ROI, but those keenly interested in ROI and vexed by the concept of paying for traffic on their trademarks still ask the question.

We’ve done some research on this topic that we’ll share below. We focused on trying to answer a couple of specific questions:

1. Do brand ads cannibalize traffic from organic listings? and
2. If the answer to the first question is: “it depends”, on what does it seem to depend?

First let’s define our terms. What do we mean by ads cannibalizing traffic?

The impact on traffic and eventual revenue of combining ads with organic listings can take many forms.

A concept that might be useful to discuss is: The incremental traffic ratio (“ITR”) of ads. That is the fraction of the reported clicks on an ad that are in fact incremental.

Let’s say that absent an ad, we expect 1,000 clicks on an organic link on a given day. Ads could have 3 general effects on traffic.

• Destructive Cannibalization: we flip on the ad and find that the overall traffic declines.

For example:

The ITR is the ratio of incremental traffic to measured ad traffic, which in this case is -200/400 = -50%

This might also be referred to as “1 + 1 = 0.5″

• Constructive Cannibalization: presence of the ad increases the overall traffic volume, but some of the traffic on the ad is ‘stolen’ from the organic listing, so the ad can’t take credit for all of the traffic it sees, only the incremental traffic driven.

For example:

Pretty clearly, Constructive Cannibalization can range from one edge case we’ll call No Cannibalization whereby 100% of the traffic on the ad is incremental, to the other edge case of Perfect Cannibalization where the presence of the ad doesn’t destroy traffic, but doesn’t increase it either.

In the language of ITR, Constructive Cannibalization exists when the ITR is between 0% and 100%.

Clearly, somewhere in the Constructive Cannibalization ITR range from 0% to 100% the advertising spend on brand ads likely shifts from a negative ROI to a positive ROI.

• Amplication: The traffic through the sponsored link is 100% incremental AND the ad is responsible for increasing the traffic on the organic link as well.

For example:The best of all scenarios, the ad creates more love all around.

When ITR is greater than 100%, an ad is having more positive impact than what is readily apparent.

Many folks with an agenda conflate Constructive Cannibalization with Amplification, using the fact of incremental value greater than 0 to imply that 1 + 1 = 3. Amplification may exist in some instances, but the existence of incremental overall traffic doesn’t imply amplification.

7 Tips For Designing A Good Test

First, we must define what constitutes a brand ad and think about whether there are different classes of brand searches. There may be some search phrases that clearly demonstrate navigational intent: they’re looking for you!

There may be others where the navigational intent is less clear: some are looking for you, some are looking for something else with a similar name. Classify these ads by type so that we may see whether the incremental value of the ads is perhaps higher when the navigational intent is less clear. Also, classify the corresponding organic listings by position on the page.

Hopefully, your organic brand links are at the top of the page, but it isn’t always the case if your brand is also a competitive search term for others (eg “online poker” is both a brand and a search term for the category).

Second, establish the measurement mechanics. You need to be able to see traffic by keyword by source (which engine). It’s best to use the same platform (attribution system or analytics package) for consistency.

Third, establish the measurement metrics. It’s best to measure conversions within X hours of a click as a benchmark. However, conversions are a much thinner data set than click traffic. Reaching statistically significant data can take much more time if one insists on using conversions rather than traffic, so traffic with an understanding of average conversion rates may get the most valid results withing the shortest period of time.

Fourth, establish the measurement methodology. Unless you actually serve the ads (that is, unless you’re Google or Bing) the only way to do this is by turning off ads and watching performance carefully. It’s best to cycle ads on and off several times to try to separate anomalous performance (I feel compelled to use the term “exogenous factors”) from cause and effect. One hour on, one hour off, one hour on, one hour off; or week on, week off, week on, week off, etc.

Fifth, make sure off is “off”. It’s best to not just pause the brand ads but to add brand phrases as account negatives to be sure that no ad is showing.

Sixth, decide how you’ll analyze the data. How will you account for expected hourly, daily, weekly, seasonal variations in search volume. A couple of good ideas: benchmark against Bing.

In other words, only conduct the test on Google, and use the Bing brand search (paid and natural) as a control. Overall site traffic can be a control as well. This is not trivial, but with sufficient data you should be able to get a very good sense of the interaction without having to hire a statistician to perform mathematical wizardry. Deciding how you will look at the data before you collect it helps keep you honest, and prevents the analyst from cooking the books to produce the desired outcome, whatever that might be.

Seventh, run the test.

We performed two such tests for different clients recently, and the results were very interesting, and not what we expected.

We expected to find that brand ads cannibalized all or almost all of their traffic from brand organic listings. We didn’t expect Destructive Cannibalization but we did think we’d see ITRs of less than 10%.

In the first test, we were stunned to find that overall 67% of traffic on brand ads was incremental. Only about a third of the traffic appeared to be cannibalized from the organic link. Interestingly, the phrases we categorized as clearly navigational did have a markedly lower ITR (55%) than those that were more ambiguous (ITR = 90%).

However, 55% ITR for navigational searches was still much, much higher than we expected to see.

The second test turned out quite differently. The data was somewhat thinner, and we’d like to have more to work with for this one, but from every angle the ITR of brand ads was essentially 0%. In this case the advertiser had never advertised on its brand, and we ran ads for a period only to find the traffic to be pretty much completely cannibalized from the organic listing.

The hypothesis we want to test is this: The more navigational the search the less likely the traffic is to be incremental. However, if outside factors make it difficult to navigate to your site, brand ads can absolutely bring in incremental traffic. In the second test, the advertisers’ brand name was very unique and unlikely to be confused for anything else. There are no other ads on the page, there are no close cousin websites that might create confusion, hence the ad provides navigational help that no one really needs.

In the first test, the advertiser’s name is closely connected with a very well known, related entity. There are many other ads on the page for brand search and many cousin links that could draw in users easily. Here the navigational assistance provided by an ad is much more necessary to make sure you get the traffic that is trying to find you.

We’d expect that advertisers with wide distribution networks like product manufacturers, entertainment, and travel will find more incremental value in brand advertisements in that there will be many other companies competing well for that traffic. “United flights to Las Vegas” would prompt ads from many OTAs with attractive offerings, not just United Airline’s ad.

Conclusion

Since N=2, here, we really can’t say we’ve done much more than establish an hypothesis for further testing. We do believe we’ve defined a repeatable methodology for studying these data in a sophisticated and useful fashion going forward and that this methodology can be extended to non-brand competitive search as well.

If 10% of sales attributed to a paid search ad are cannibalized from an organic link, doesn’t that change its ROI? If, in non-brand search, ads and organic links amplify one another, isn’t the ROI better than what it appears to be? Shouldn’t we act on that information?

One conclusion we can draw definitively is: answers lie in the data, not in marketing blather. Average values are meaningless as each case varies. If you wonder about whether you should or shouldn’t advertise on your brand there is only one way to find out.

The statistics to tease out exogenous effects ‘properly’ is non-trivial, but don’t let the perfect be the enemy of the good enough. You can get pretty close to the ‘right’ answer with smart application of basic math.

Happy Testing!

Given that my last post for SEL was on Quality Score and suggested that landing page quality was mostly a hammer used to beat up bad actors and didn’t have much meaning for legitimate businesses, this announcement was a bit embarrassing. It also made me curious as to how quality will be defined.

I got a chance to speak with Jonathan Alferness, director of product development on the Google ads quality team, to delve deeper into the purpose of this initiative and try to glean insight as to how landing page quality would be measured.

Jonathan was very helpful with respect to the former question and understandably protective when it came to the latter.

Below, I’ll make a clear distinction between what Google is willing to say publicly about landing page quality and what I believe this means. I’ll summarize what Google has said either through my conversation with Jonathan or elsewhere in print in italics. My speculation will be in normal font.

Why Did Google Do This?

Jonathan made the case to Pamela and to me that the intent is to improve user experience.

When landing pages don’t line up well with the user’s search, the user has a poor experience, and ultimately that’s bad for everyone.

It’s bad for the user because they don’t end up where they expected to end up; it’s bad for the advertiser because they clicked on an ad and ended up somewhere they didn’t want to be, winning the advertiser no great brand impression; and it’s bad for Google, because that user is more likely to seek a different search experience and less likely to use Google sponsored links in the future.

This makes perfect sense. As I argued in last month’s post, Google maximizes its immediate term revenue by making QS 100% based on anticipated Click-Through Rate (CTR). Factors related to landing page or anything else reduce Google’s revenue per SERP view by reducing the importance of CTR.

However, poor landing page experiences might reduce Google’s long-term revenue by training users not to click on sponsored listings. That could jeopardize the business.

By creating a finer gradation between “your landing page stinks” and “it doesn’t stink” — essentially where we’ve been with landing page QS — Google rewards advertisers that pay attention to landing page decisions and creates another incentive to provide a great user experience.

In the long run, better landing pages lead to higher conversion rates, which in turn means an increase in the value of traffic, therefore allowing higher bids and correspondingly more traffic at the same efficiency. It’s a beautiful virtuous circle.

It’s also at least part of the reason that Google bought Urchin and made Google Analytics free. Giving advertisers the tools to diagnose user behavior and improve website effectiveness should ultimately lead to better monetization of traffic and higher bids in the paid search auction. Smart. Very smart.

How Much Weight Will Be Given To Landing Page Quality?

More.

Google isn’t saying much on this topic, and what they’re saying is a bit of a mixed bag. Jonathan told Pamela that ads with high-quality landing pages will get a “strong boost”; he told me it would be a “slight boost”; and on the AdWords blog post it says: “We expect most campaigns will not see a significant change in overall performance.”

They began the roll-out in South America a month ago and appear to be rolling it out to smaller accounts in the US first. This suggests to me that they think it could have a material effect that they want to watch very carefully. Perhaps not.

Making even minor changes to software should be tested thoroughly before roll-out. Perhaps most campaigns won’t see a significant effect because most landing pages are roughly equivalent, so this change will mostly impact the “outliers”?

It’s hard to imagine that Google will sacrifice revenue on this, either. They’ve likely done the math to see how much gradations in landing page quality affect the propensity of people to use sponsored links in the immediate future, and have tuned this dial to be revenue positive even in the short run. It will go on to be yet another one of Google’s revenue dials, subject to periodic “tuning.”

How Will Google Determine ‘Quality’?

Jonathan said: “Google is focused on both relevance and content on the landing page in addition to the user response to such landing pages.”

I pressed on this with the question: “Will advertisers with Google conversion tracking on their sites have an advantage over other advertisers, given that conversion rates are a very clear signal as to whether users found what they wanted?”

His response was: “There is no special information derived for some customers but not others. We apply the principles of landing page quality and relevance to every ad and every landing page. The advertising system is dynamic, as an auction takes place for each query.”

The behavioral signals that Google traces must therefore be related to what they can track for all advertisers, not just those with Google conversion tracking: bounce rate, propensity to search again in short order, time on site, etc. I’d love to know how much of this behavioral data is gathered through Google toolbars as a sampling method rather than from watching browsers return behavior from Google.com, but I don’t know that.

With respect to the relevance and content on the page, Google means to determine: Does this page match the user’s intent? Behavior is one guide, but content on the page will matter as well. Pages that seem to be about the topic of the user’s search will do better here than general landing pages.

It would be surprising if something as simple as blowing the user’s search string or the keyword back at the user dynamically on every page would be meaningful in terms of content on the page unless it affects user behavior.

I suspect the advertiser will be rewarded for landing a “ball peen hammer” search on a page with ball peen hammers on it, rather than a page with only wrenches.

High quality landing page for “ball peen hammer” search:

Previously high quality landing page in Google’s eyes — ie: “not spam, and loads fast” — now considered low quality content:

Another High quality landing page for the same search: I bet they like a product page just as much

Whether the advertiser selects the ideal ball peen hammer page chosen among several options is probably beneath the level of detail they’re getting into at this point on the “content on the page” piece of the equation.

I’d be stunned if Google tried to differentiate between the first and third based on content on the page, but our experience suggests that the conversion rates will probably be higher for the first page.

One worry that I expressed was that Google would bake in some preconceived notions about good pages and bad pages.

For example, we’ve heard it expressed that Google thinks taking someone from a Google SERP to someone else’s SERP is a bad experience, therefore a website’s search results pages might be frowned upon as a landing page choice.

Would Google arbitrarily say this was a poor quality landing page for a “ball peen hammer” search?

That would fly in the face of our experience, which suggests that often a good site-search results page outperforms a well-chosen sub-category page in terms of conversion rates.

Similarly, a “view all” page often outperforms landing folks on page 1, but it’s conceivable that latency considerations due to loading “all” the images will trump what we know to be better performance from a conversion rate perspective.

Google could also conceivably decide which page on an advertiser’s site ranks highest for them in organic listings on a given search and reward folks who choose that page for paid search ads as well. That would rankle those of us who believe smart humans who pay attention pick better pages than Google does. More on that in a later post.

Jonathan responded that Google wasn’t baking in preconceived notions, that relevant content and user behavior would define the landing page quality.

Conclusions

Within e-commerce, this could serve as one more penalty for those who aren’t mass merchants. If you sell office chairs of the $500 variety, not the $40 variety, you’ll likely find your conversion rate on the term “Office chairs” to be worse than a “competitor” who sells the $40 version even with qualifying ad copy to wit “Starting at $499.”

The conversion rate is lower, the bounce rate is higher, so behavioral signals after the click incur a landing page quality penalty on top of the CTR penalty incurred by the qualifying ad copy. This double whammy may make it even tougher to compete.

Maybe that’s the “right” result for users, but maybe it takes away diversity of choices for those who actually read the copy.

At the end of the day, it appears likely that this change will primarily reward advertisers for doing what they should have done all along. Pick the best page among the available choices for your PPC landing pages, and design those pages for maximum effectiveness in monetizing traffic. If you’ve been doing that since day one, expect a bit of a boost in traffic in the coming weeks.

(Stock image via Shutterstock.com. Used with permission.)

The exact recipes for Quality Score on Google and Bing are unknown, and it is that uncertainty that creates so much room for mischief and blather. Uncertainty causes some smart people to rack their brains trying to figure out what’s in the black box.

Other folks in the scoundrel category seize the opportunity created to present themselves as the QS Master who knows the secrets that can lead to riches. At least one such pretender claimed that instead of managing bids he preferred to manage through QS manipulation. Aye Carumba!

The name itself allows some misguided folks the opportunity to create garbage ‘diagnostic’ tools that identify ‘opportunities’ to ‘optimize’ one’s account by identifying ‘poor quality’ ads.

Let’s clarify a few things.

According to Google, Quality Score is Calculated from:

• The historical clickthrough rate (CTR) of the keyword and the matched ad on the Google domain; note that CTR on non-Google sites (such as AOL.com) only ever impacts Quality Score on our search partners – not on Google
• Your account history, which is measured by the CTR of all the ads and keywords in your account
• The historical CTR of the display URLs in the ad group
• The quality of your landing page
• The relevance of the keyword to the ads in its ad group
• The relevance of the keyword and the matched ad to the search query
• Your account’s performance in the geographical region where the ad will be shown
• Other relevance factors

CTR

We can essentially combine the first three bullets with the ‘relevance’ and geography bullets to get: Predicted Click-Through Rate. This is the most important piece of the puzzle.

When you look at the math, Google maximizes its revenue per SERP impression when it considers only bid and CTR as factors. Therefore: Any component of QS that diminishes the role of CTR in this equation loses them money, at least in the short run.

The engines aren’t going to swing too far away from CTR as the ultimate measure of quality, because doing so costs them money.

Landing Page

If the landing page provides a lousy experience the user will have a bad impression of not just that site, but the experience of clicking on ads as well. Short term, this doesn’t impact the engines, but in the long run it certainly could.

What landing page provides a lousy user experience? One that is slow, one that is unrelated to the search term, one that is chock full of ads.

If you run a legitimate business and the ads link to the right pages on your site, and the pages load reasonably fast, you’re home free. Moving pixels around on the page isn’t going to lead you to QS Nirvana.

That’s all there is to it.

So if it is really all about anticipated CTR, why might my CTR be low?

It’s important to create tight Adgroups and write good copy. Well worth the time to do this right.

But here’s the thing: when people see a low QS they often mistakenly assume the reason is number 1 or 2, when in fact it’s often 3, 4, or 5.

People who don’t understand this waste countless hours trying to find the magic grouping and magic wording to get them 10′s. In many instances, there isn’t anything you can or should do to “fix” the “problem”.

Qualifying Ad Copy

Let’s say your business sells high-end jewelry. The mass market jewelers will tout discounts and “Ruby earrings starting at $15…” Your earrings start at $300. You’ll have a higher quality score if you tout discount percentages and offers, but doing so will torpedo your conversion rate.

It’s likely better to qualify the traffic with “Starting at $300″ to weed out the discount hunters, but doing so will hurt CTR and therefore QS. Testing will reveal which makes the most sense. As I’ve argued previously, this game favors the mass marketers over the high-end merchants.

User Search Is Ambiguous

Someone searches for “Yamaha”. You sell Yamaha keyboards, but you don’t sell motorcycles, or boats, or stereo equipment. In all likelihood, unless you’re Yamaha corporation, your QS is going to be awful, because some fraction of people typing that query are looking for each of those  and the QS for merchants of any of the single categories will have poor CTRs.

Low Commercial Intent

Someone searches for “Central Park”. You own a hotel near Central Park, but your QS on that keyword stinks. In all likelihood, this is a function of the fact that most of the people typing that phrase aren’t looking for hotels, trips, restaurants or anything else related to commerce.

Maybe they’re looking for a map, or directions, or history, but the CTR will be lousy because too few people typing that search have an interest in any specific ad.

The fact that the quality score is low doesn’t mean the ad text is sub-optimal and it doesn’t mean you can’t profitably advertise on that keyword!

People worry far too much that having low QS keywords in the account or adgroup will hurt the QS of other keywords. The engines have a huge  interest in preventing that from happening.

Anticipating CTR accurately is a fundamentally important activity for them tied directly to their bottom lines. They’re going to get this right because it matters to them, and they have a ton of really smart people working on how to get it right.

Again, poor wording and poor grouping absolutely can be problems and should be addressed, but too much time is spent applying medicine to keywords that aren’t sick.

3 Request For The Engines That Would Help Us All

1.  The Reset Button

We’ve inherited accounts that have been poorly managed historically and have to fight an uphill battle initially to overcome poor QS history. Setting up a new account is somewhat of a hassle and doesn’t address the piece of the problem tied to the advertiser’s display url.

It would be great if there was an “erase history” option that could give the new management team an opportunity to right the ship quicker. Historical data is often a good predictor of the future, but in cases where new management has taken over, the history can be a lousy predictor. Getting the CTR predictions right benefits the engines and the high quality managers and advertisers.

The challenge is that bad actors will want to hit that button every day to start over, so I propose that each advertiser can only hit that button once a year. Maybe only the engines can hit that button based on a verified and reasonable request.

2.  End the promotional event penalty

Some advertisers have promotional events that they want to tout in the ad copy, but we find that changing copy frequently incurs a penalty because the engines have no history to go on with respect to the new Keyword-Copy combination so they erase the good history and start you back at square one.

By the time the short-term promotion is over, the engines may not have figured out yet that the new copy actually boosts CTR. This is a tougher engineering nut to crack. We can’t reasonably ask the engines to always assume a change is for the better because in a transition from control copy -> promotional copy -> control copy, the second transition likely hurts CTR.

Perhaps copy blocks can be deemed promotional or control. The engines could then track the average CTR impact of transitions from one to the other and make better assumptions about the CTR of a new change based on that history. Those who try to game the system will have data trends that indicate that there is no historic difference between the two, so no help.

3.  Give us a scatter plot

In a big program, it’s tough to tell whether CTR and QS are low because of the copy (#1, 2, or 3 above), or because of the user intent. One good way to know would be to reveal a scatter plot of the QS for all the ads in the top 10 positions.

Without identifying which QS went with which advertiser, we could still get a sense of whether we’ve done a poor job of matching keyword to copy, or whether no company scores well on this particular Keyword.

For example, the plot below would tell us that different copy could help QS. It may not be wise for us to ‘fix’ this copy if it qualifies traffic, but at least we’d know it’s an issue.

This plot would tell us that no advertisers have a great score on this keyword and we’re likely wasting time trying to ‘fix’ a problem that doesn’t exist.

Some of us geeks would prefer a numerical value like standard deviation rather than graphical representation, but perhaps both would be ideal.

We all benefit from working towards better performing programs. These changes will help advertisers and engines alike, by getting better predictions of CTR and helping advertisers identify where to put their finite management resources.

We get really excited anytime the engines give us new information to analyze or tools with which we can fine-tune our approach.

Google has just given us a treasure trove of new information: click level data on an ad’s position. In the past, we’ve had to rely on observations and average position day to day to get a sense of where an ad is appearing on the page. But we know that averages lie.

By giving us information identifying the position on the page for each click we get a whole new window on auction dynamics.

How It Works

Similar to the Google Click id, gclid, by flipping a switch in your adwords account, Google will append an ad position parameter called ‘adpos’ to your destination url. We can then parse this information as it’s passed to our clients’ sites and analyze from there.

The values passed in the adpos parameter reveal the page of the results off of which the ad was clicked, whether the ad appeared on top of the organic listings or to the side, and its overall position rank on that page. All of this is passed in one text string.

• 1s3 means: page 1, on the side of the page, third position among ads.
• 2t1 means: page 2, on the top of the organic listings, first position.

What It Reveals

Matt Mierzejewski, our VP of Paid Search, flipped that switch immediately for one of our clients to start gathering data, and I couldn’t resist digging in, even though we only had a couple of days worth of data.

To wrap my arms around this, I narrowed my study to 8 ads representing 4 different keywords to reveal among other things what impact match type might have on ad serving.

Pages

Of the 471 clicks in my sample: 97% happened when the ad was on page 1; 2% on page 2 and less than 1% on page 3 or more.

The academic in me wonders: if an ad is in position 2 on page 2, how does that figure in the average position calculation at the end of each day? Does that count as position 2, or position 12 or whatever it might be given that the page 1 ads were ‘above it’? I suspect the former, but in truth given that 97% of the time we’re looking at page 1, it probably doesn’t matter.

Position

Fascinating, fascinating, fascinating!

Keyword 1. Adwords reports that Keyword 1 on broad match had an average position of 3.7

But take a look at the positions from the perspective of click traffic:

This is just jaw-dropping, IMHO!

The Average Position reported is 3.7, but the weighted average of the clicks by position is 2.6. Bear in mind, the reported Average Position is based on impressions served, not ads clicked. The click-wise view will always be a lower ordinal number reflecting higher position on the page. This is a product of the higher click-through rates of higher positions.

That said, for a reported average position of 3.7, 67% of the clicks came when the ad was in position 1 or 2!

Boy-oh-boy, do averages not tell the whole story!!!

Let’s take a look at Keyword 2, also on broad match. Average reported position for the days covered: 4.9

Even more dramatic! The average position of ads clicked is under 2.1!

These results are so startling to me that I wonder if I’m comparing apples and oranges. The average position data comes in day long chunks, and the click data I’m looking at starts midday and ends midday.

I also wonder if Google appends the parameter for network partner clicks…I think the answer is yes, but…Also worth noting that our own day-parting activities could be driving this as much as auction dynamics.

This view of data will have us all scrambling to revisit what we thought were settled questions about paid search. One obvious example: we long ago concluded that conversion rates don’t appear to vary much by position. I’d be surprised if we found variance given the new view.

Hal Varian’s study, with the crystal clear view of the auction that only available to Google, suggested that conversion rate is relatively invariant with position, and that’s actually somewhat contrary to Google’s best interest — they’d benefit from saying higher positions on the page convert better. Still, the new visibility means we can and should take another look.

Placement

For ads that are primarily in position 1, what fraction of the clicks come from impressions when the ad is on top of the organic listings versus on the side?

Keyword 3 on broad match had 114 of its 140 clicks placed when the ad was on top of the organic listings. The same keyword on exact match saw all 112 of its clicks on top of the organic listings.

This seems to be the trend. There is somewhat less variance in both position and placement when the ad is on exact match than when it’s on broad match. That makes sense, as the competitive landscape is likely different for every broad match variant.

It’s worth pointing out that we don’t get a perfect view of the page layout from this.

We don’t know whether 1s3 means the ad was third from the top on the right, or at the top of the right column with 2 ads served over the organic listings, or whether it was below Product Listing Ads. Nevertheless it gives us much more information at the click level than we’ve had before.

So What?

Well, that’s a hard question in this case.

What are we going to do with this data? I don’t know…yet. Perhaps it will shed light on how national ads compete in various local markets?

We may be able to see using geographic overlay data that clicks from certain locales are coming from significantly lower on the page than the national average, indicating stiff competition from local brick and mortar stores. That might suggest creating a geo-targeted campaign for that region with separate bid logic and copy to provide a competitive edge.

That’s just one idea. The point is that every time Google has given us greater visibility and greater flexibility the leaders in the industry have been able to figure out ways to raise the performance bar.

I expect this to be another opportunity for differentiation.