Ad Testing: Are You Using The Wrong Success Metrics?

Everyone knows you should test ads – that’s not debatable anymore. However, what is debatable is which of the ads you are testing is actually the best ad.

I recently polled a large group of marketers and asked them of these ads, which one performed the best:

Take a moment to decide which you think is the winner.

Here were the responses:


Just 7.3% thought there wasn’t enough data to determine a winner. Those are the ones who are correct.

The Current Metrics Are Flawed

When you consider all the stats needed to determine a winner; you need to work from these numbers:

  • Impressions
  • Clicks
    • CPC
    • Cost
  • CTR
  • Conversions
    • Average sale amount

From those metrics, the most commonly generated stats to determine a winner are:

  • Conversion rate
  • CPA (cost per acquisition)
  • ROAS / ROI
  • CPI (conversion per impression)

Each one of those metrics has a fundamental flaw.

  • Conversion rate does not take into account impressions, click through rate, or average sale amount.
  • Cost per acquisition does not take into account volume (impressions and clicks) or average sale amount.
  • ROI / ROAS does not account for volume (impressions and clicks).
  • Conversion per impression does not take into account average sale amount.

It Starts At The Impression

Assuming your campaign is targeted towards sales (we’re going to leave the branding discussion for another day); you should be measuring from the impression.

You choose a keyword. Someone searched for your keyword. That combination creates an opportunity for your ad to be displayed. It also creates an opportunity for a conversion.

Yes, every single impression has an opportunity to generate a sale. Therefore, your measurements should start at the impression as well.

Clicks Through Rate vs. Conversion Rate

One of the problems with testing is that you have two big metrics you’re often examining: CTR and conversion rates.

However, which of the scenarios below is better?

  • High CTR, low conversion rate
  • Low conversion rate, high CTR
  • Mid range CTR, mid range conversion rate

It’s an impossible scenario to measure unless you are using a metric that incorporates CTR, costs, conversions, and average sale amounts into a single number. That’s what Profit per Impression will do.

Profit Per Impression

I first started using this metric in 2002 and first started writing publicly about it in 2006. I’m still amazed at how little adoption this metric has since it is the overall best metric to use for testing and its very easy to calculate.

To calculate this metric, all you need to do is take the original stats from above and add a few pieces of data:

  • Revenue (this might take some effort if you are doing ecommerce with variable sales amounts and product costs, but it is worth the effort)
  • Profits (simple: Revenue – cost. I’m ignoring the complex aspects here of adding lighting, salaries, etc)

With those metrics, calculating PPI (profit per impression) is easy; just divide profit by impression:


It turns out the ‘best’ ad is not the one with the lowest CPA, the highest CTR, or the highest conversion rate.

It turns out that 4.2% of the original voters were correct (or 11.7% if you count those who voted that there wasn’t enough data); but since they didn’t have the correct data it was most likely due to a good guess.

The one issue with taking stats from Google ad tests is that Google does not rotate ads evenly; but there are some rotate fixes available). To prove the PPI theory, let’s see the extrapolated stats as if every ad actually received the same number of impressions:


As you can clearly see, ad 1 is a clear winner in terms of total profit.

If you happen to be wondering why these stats seem so high on the revenue basis; its because the revenue is not based upon a one time sale. It’s the lifetime value of the customer. Always take into account lifetime values. Here are the stats if you base the numbers off of just first month revenue:


While ad 1 is still the clear winner; from a single month basis the highest CTR ad is actually losing money and doesn’t appear to be profitable.

To get a clear picture of your actual revenues, if customers buy from you more than once a year, which includes subscription products, then you need to calculate PPI from the entire customer’s revenue and not just the first time sale.

The Quality Score Wrinkle

There are some times when I won’t actually pick the best PPI ad; and that is often due to Quality Score. If the second best PPI ad has a much higher CTR than the best PPI ad, then I will often pick the 2nd best.

Because CTR is such a huge quality score factor, if you pick an ad with a much lower overall CTR, you could cause your Quality Score to drop which in turns means you average position will decrease or your average CPC will increase (or both).

If in this test, ad 4 (with a CTR almost double all the other ads) didn’t have a $0.39 Profit per Impression, but was instead in the $0.43-$0.45 range; then I would have declared it the winner instead due to the Quality Score wild card.

Alternatively, I would have set up an ACE (AdWords Campaign Experiments) test between the two ads to see if I could get some Quality Score insights between the two ad and keyword combinations. However, it was enough lower in total profits that its not the winner by a large margin; and therefore, I would pick ad 1 as my true winner.


Testing is easy. Pick at least one ad group and:

  • Write a few ads
  • Wait
  • Measure the results
  • Learn from the losers
  • Delete the losers
  • Write another ad or two
  • Hit save
  • Wait
  • Measure
  • Repeat

There’s nothing difficult about testing. However, if you are using the wrong success metrics in your test; then your time and efforts will be wasted. By using Profit Per Impression metrics, you can be assured that you are pick the ad in your tests that will bring you the most profit for your paid search account.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: SEM | Paid Search Column


About The Author: is the Founder of Certified Knowledge, a company dedicated to PPC education & training; fficial Google AdWords Seminar Leader, and author of Advanced Google AdWords.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Pat Grady

    reading it, i’m thinking of CTR and its QS impact…  when i finally got to your “The Quality Score Wrinkle”, i confirmed (again) why i’ve always told my crew that you are the best there is.  kudos on another brilliant article BG.

  • GardnerDennis

    my co-worker’s sister got paid $21912 the previous week. she gets paid on the internet and got a $416800 house. All she did was get fortunate and put into action the steps given on this link

    ⇛⇛⇛⇛► (Click At My Name For Link)


  • Bart van Duinkerken

    Definitely an article i will come back too. I like the metric PPI never heard of it before but is actually a great metric to use, together of course with the CTR

  • Phil O’Doherty

    “However, which of the scenarios below is better?

    High CTR, low conversion rate
    Low conversion rate, high CTR
    Mid range CTR,Mid range conversion rate”

    Are the first two options not exactly the same?

  • Brad Geddes

    Phil – Doh! You’re right. It should be:
    High CTR, Low conversion rate
    High conversion rate, Low CTR

  • Mark Welch

    I’m not sure your first chart/table is what you intended (is this really what you showed the audience you surveyed?).  And I disagree with “PPI” as the most important metric. I’ve re-created your chart and posted my comments on the concept at 

  • Ted Ives


    Saw your posting – good addition to the chart, ROAS is a smart goal to shoot for often.  Net margin or LTV goals of course are even better.

    But it’s not like your goal exists in a vacuum, the real question is:

    What is your goal…and what constraints is it subject to?

    For instance, profit maximization makes sense if you have an unlimited budget.  On the other hand, if you don’t have enough budget to implement Ad #4 everywhere,  then being a good steward with your budget and maximizing ROAS may prove a more viable option (thus Ad #1 makes more sense then).

    Optimization of any factor in just about any walk of life is almost always subject to constraints of one sort or another, the trick is figuring out what they are.

    For instance, I suspect Brad’s constraint for this article was SearchEngineLand’s maddening graphic size restriction, he probably would have included ROAS in his chart otherwise ;-)

  • McintoshClifton

    my buddy’s sister-in-law made $18108 a month ago. she worrks on the internet and bought a $525400 condo. All she did was get blessed and put into action the instructions given on this website 

    ⇛⇛⇛⇛► (Click At My Name For Link)

  • mateo meier

    Well done post! Keep up the good work.

    Here in switzerland google started offering free seminars (half day
    seminars) to manage PPC campaings. Thank you for your valuable post


  • Dan Perach

    is there an ad tool you would recommend Brad?  i’ve used loweryourbidprice tool and it was great, but only for ctr, not ppi. is their a tool out there for ppi?


Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide