Why Changed Goals Require Changed Metrics

Enterprise businesses demand savvy communication from SEM managers. It is the nature of large organizations to have the following:

• Senior executives reviewing data from programs they don’t understand
• A shortage of institutional memory
• Consultants coming and going who offer opinions based on superficial data

Each of these traits make performance reporting particularly important, and sometimes challenging, for paid search. These challenges are often most keenly felt when the enterprise decides to make significant changes to the goals of the program.

Any well-managed paid search program* is subject to the law of diminishing marginal returns. The last dollar spent has a lower ROI than the first dollar spent because we buy the most cost-effective advertising first. Fans of RKG Blog are probably tired of hearing about it!

*Note that a badly managed program may not be subject to the same law. Improving volume and efficiency simultaneously can happen when poorly managed programs are placed in the hands of experts with great tools.

What this implies is that there is an inherent tradeoff between volume and efficiency at any point in time. Making the program bigger today will also make it less efficient, all other factors being equal.

Enterprises Change Goals

Business needs, changes in management, competitive pressures, or any combination, can lead to significant changes in paid search goals. Where the enterprise was using paid search for brand awareness and customer acquisition and was happy to invest in future growth, now the goal shifts to making search profitable in the immediate term, or vice-versa.

The beauty of paid search is that we can turn on a dime to hit those new goals.

The problem is that flawless execution will produce “bad” year-over-year comparisons that are a natural consequence of the change in goal. Senior executives in other departments and consultants (having no institutional memory) will proceed to beat up the paid search manager for doing her/his job well.

Intrepid SEM Manager Pat: “Last week paid search drove $1.5M in revenue at a 5:1 ROI”

Angry Memory-Challenged Executive Shawn: “YOY, paid search was down 15% last week, why do you suck?”

Pat: “Last year the ROI target was 2.5:1, so we had to pull back to achieve the 5:1.”

Shawn: “I’m not interested in excuses, I want to know why every week we’re down in paid search?”

Pat: “We’re doing really well! We’re twice as efficient, we just can’t match the sales volume at that efficiency.”

Shawn: “You call being down 15% doing well’?!? Shape up or ship out!”

Managers want to see graphs that go up to the right; they don’t want to see graphs that go down to the right (positive slope = good; negative slope = bad). In YOY comps, negative numbers in red are bad. The fact that those negatives were caused by other metrics turning in a positive direction is irrelevant to Shawn.

In fairness, these folks can’t have expert understanding of every number that crosses their desks. Perceived declines require explanation. That’s just managers being managers. The confusion isn’t the fault of Shawn; it’s our fault for not providing the right metrics on which to judge success in the new world order.

Problem: Excellent Performance Looks Bad

The real problem above is that the goals of the program shifted dramatically, but the metrics used to gauge its success did not.

In the case above, the enterprise clearly decided to move from a top line emphasis on driving revenue to a bottom line emphasis on driving profits, but the organization kept looking at the top line revenue numbers. Declines there looked like failure.

Let’s take a look at a sample data set to highlight the effect.

The above data from a fictional e-commerce website spending to a 2.5:1 ROI target for a year and 19 weeks, then shifting to a 5:1 target. For sake of argument, I assumed this would roughly cut the spend by 50% and the sales by 25% (that Law of Diminishing Returns notion).

For executives watching YOY performance of Revenue and ROI, they’ll certainly see the ROI improvement, but a decade of experience in the space suggests that they’ll nevertheless focus on the YOY revenue decline. “Wow, we were up 20 – 25% YOY, now we’re down?!?”

The YOY revenue data will look particularly bad.

Each week, the message will likely be: “Good job on the ROI, but what are you going to do about the revenue? We really need to get those numbers back up.”

Experience suggests that that conversation will get old after a fairly short time. Moreover, it causes the execs to second guess their own decision making in ways that don’t necessarily make sense for the business.

The Solution: Change The Metrics

Presumably, the change in goals reflects a realization that the program had been losing money, and a decision that it now needs to make money. Whether that is the right business decision or not, if the objective of paid search is to be more profitable, then profitability is the metric we should track.

A good metric to use here is Marketing Income. In e-commerce Marketing Income is defined as revenue minus cost of goods minus variable costs associated with credit card fees, pick-pack, etc., minus advertising costs.

If we assume an average margin of 40% for this company (obviously better if we can get the actual margin down to the KW level, but we’ll take what we can get), we have the following:

The wise SEM manager will say when the goal change is announced: “Terrific, we’ve decided to focus on profits. Let’s recast our reporting to show Marketing Income by week, and YOY Marketing Income improvement.”

Recasting the exact same data shows the following:

Note that instead of sending mixed signals with one metric going up (ROI) and one going down (Revenue), both metrics make a sharp turn in the desired direction. This isn’t an effort to hide the facts, it’s an effort to focus attention on the metrics we’re supposed to be improving and showing that, in fact, they’re improving.

But What About Lead Gen?

Suppose instead of e-commerce, we’re talking about a site that uses lead capture as it’s best metric of success. Let’s say this lead gen company has been willing to spend $25/lead, and now realizes that it needs to cut that figure to $15/lead.

If they’ve historically averaged 10,000 leads per week, that figure will obviously drop with more restrictive efficiency targets.

Yes, the CPL line in red drops, which is good, but so does that Leads per Week figure, and that will cause consternation unless the savvy search manager can change the conversation to better reflect the positive performance improvement.

The exact same logic we used with revenue to marketing income applies if we make a couple of assumptions. We may not have dollar figures and margin to reference, but if we make the simple assumption that the change in goals was precipitated by a realization that the leads weren’t worth $25 a piece in the past, we can plug in a value of what we do think they’re worth and get to a marketing income graph.

Are Senior Managers So Daft That This Switch Will Work?

Daft isn’t the right word. Busy, preoccupied, and unable to understand the details of every program is the more accurate. And, yes, controlling the discussion and focusing on the right numbers makes a huge difference in perception.

Is This Deception?

Are we hiding the truth from senior management when we do this? Absolutely not. We are focusing attention on the true goals of the program as they are today.

Driving great results is one part of the paid search managers job. Communicating those great results well to upper management is often every bit as important. Speaking truth to power is critical. Doing so in a way that accurately reflects the results of your work is wise, particularly if you do good work.