Attribution Wars: A Plea For Small-e Enlightenment (Part 2)

Pssst. Over there, next to the purple wing chair. Behind the foosball table and just beyond the 2002 vintage lava lamps. It’s a gaggle of digital marketers, munching on Pez, and screaming at the unfortunate Fortune 500 brand CEO who made the mistake of visiting this digital agency’s hip warehouse space conveniently located just steps from an eight-lane highway. “We want credit! We want credit!,” the digerati are chanting over the roar of trucks and stretch limos. A few, more impatient than the rest, begin pelting him with M&M’s. He’s being very slowly stoned to death. A member of the agency’s management team, clad in yoga pants and clutching a venti chai latte in one hand, uses amazing grip strength in her other forearm to bombard the stunned client from the left flank, with a powerful t-shirt cannon.

Sound realistic? Didn’t think so.

The fact is, digital marketers don’t have to tug on the CEO’s pant leg as much as they used to. And definitely no need to pelt the client with candy. A Forrester Research study for 2009, “The State of Retailing Online,” conducted in conjunction with Shop.org, continues to show paid search outstripping all other channels for perceived effectiveness. A whopping 83% of online retailers (including many of the largest) rate paid search as an “effective” acquisition channel, far ahead of #2, organic traffic, at 51%. “Offline advertising” only gets the nod from 20% of this group. “Banner ads” still draw derision. Only 9% of online retailers believe they’re effective in driving acquisition.

Little wonder that frightened spokespeople for the impact of offline spending and display ads on the effectiveness of that beloved paid search channel feel the need to circulate straw man arguments that accuse our side of skewing attribution.

For a relatively new channel, paid search is doing too well for the liking of traditional advertising people. The average large online retailer spent $5.9 million last year in the channel; that’s nearly three times what they spent even emailing their house list. Relative to their size, midsized companies in the space probably spent the most impressively, at $3.6 million. “Little guys” in the survey checked in at an impressive average of $202,000. A lot of these companies don’t spend a dime unless the cash registers are ringing as a result. If they’re paying for these clicks, chances are they’re working.

Sadly, some observers see this as a zero-sum game, and that’s one of the reasons we’re really debating attribution today.

Why this debate? Why now?

What type of knowledge do we have about our advertising investments, anyway? What type of knowledge? What? Huh?

Epistemology. You’re probably thankful you haven’t heard that term since undergraduate philosophy class, and not too pleased to be hearing it now. (I never took that class.)

But that word may be needed if we’re to understand what’s going on with today’s sudden interest in attribution.

There appear to be three major reasons people in our industry suddenly want to study more closely how to attribute “credit” to different media sources:

1. They’re selling all-encompassing solutions that promise to correctly explain every possible influence that goes into a consumer’s decision to buy;
2. Like the imaginary, immature digerati in the fake scenario of the opening paragraph, they want to claim “credit” to draw more budget to themselves in a zero-sum budget allocation game;
3. They just want to get a little better at optimizing their campaigns in their digital specialties, playing “small ball” to hit singles and doubles, and score predictably for their companies and clients.

Epistemologically, they’re all pretty far apart. #2 ranks at the level of the trivial interest-seeker. Economists call it rent-seeking behavior; political scientists call it “interest group politics”. Been there, seen that. Move on. There’s no argument here. Just self-interest.

Camp #1 is potentially very influential – and I worry that the well-meaning hope to unravel all the mysteries of response science will derail working marketers from pursuing the small successes that are within their reach.

On that: Grand Narratives have been out of favor for half a century or so, for good reason. When modernity started to stick in the craw of clever (mostly German or French) commentators in the last century, it wasn’t that we wanted to chuck science. It was that the “totalizing” narratives of capitalist and communist empires alike took things too far, claiming to explain everything.

In a world where the sizes and shapes of organizations keep changing and a steady stream of data serves to remind us that there is possibly more complexity to the economy than we can hope to tame, many of us have adopted an attitude to “take it as it comes”. It’s probably important to understand how a whole system works if we’re talking about global climate change, or DNA research. But outside of a few Dr. Evils at the largest of companies, depending on your job description, understanding how the whole system works is relatively unimportant in marketing. Doing well on the component parts can be very effective in creating more total profit.

This attitude may explain why a lot of people quietly admitted “me neither” when the unfashionable George H.W. Bush said: “I just don’t get the vision thing.”

To sum up: I’m proposing – for most of us – a sort of Goldilocks approach to this. Don’t claim to understand everything (too much science), but don’t just show up to the debate because you want more money for your department (spiteful science). We have, then:

(1)    Too Cold: The folks who want perfect attribution based on an all-encompassing system, impeccable measurement of all factors, even if – in order to accomplish their brave project – they need to invent tracking solutions that take data collection to new levels that surely violate the privacy expectations of users, if not the sheer letter of the many individual “Terms of Service” they agree to.

(2)    Too Hot: The hotheads who argue for more credit just for their own specialty, and less for the other guys, in a naked show of realpolitik. Today’s worst violators: the traditional agencies, engaged in a campaign called Defend the Spend. At all costs. And if it’s a zero-sum game, they’ll Defend the Spend even at the expense of emerging digital campaigns that (in our opinion) are doing very well, thank you. Are they right? We don’t think so. And some of us have t-shirt cannons and an attitude to back it up. Let’s hope we actually have numbers, too.

(3)    Just Right: The pragmatists who really just want to study the issue a bit more and solve their company’s or client’s digital media attribution problems a little better than they were being solved last year.

The data revolution is here to stay (but it has its limitations)

It was easier when we were mostly farmers, wasn’t it? No “multiple points of attribution.” Farmers could debate whether it was the weather, or someone’s work ethic, that contributed to crop yield that year. But it was mostly the weather.

Hold on, though. Farming’s probably no different from many other fields. It, like all the others, is suddenly going to be revolutionized by a wealth of data, and intervention from pesky outside number crunchers.

The mythology is already spreading. “The old baseball men” and their instincts and superstitions: replaced by statistics nuts. Moneyball.

Imperfect professional wine ratings by swishing, slurping, snobs: under threat from a Super Crunching computer model that predicts wine quality for a given year and region from a host of climate factors.

Weird bets by Hollywood studios, sometimes leading to expensive flops, are under threat from hedge-fund-backed Relativity Media, a production firm that has now backed hundreds of good films (though if you listen to its critics, few or no great ones). Relativity is known for its extensive number-crunching to bet better on which films will make money (and avoid losing it), but it can’t be accused of claiming to have all the answers.

Needless to say, we have to get better at number crunching in many fields, or we’ll be pushed into obsolescence, unless we have the budget of the New York Yankees.

But if it’s fashionable to do more number crunching, it’s also hard to tell the difference between the real deal and the snake oil salesmen.

Back to digital marketing. Despite the apparently huge differences in motivations to approach attribution from different angles, we can agree on this: attribution is broken. We know this much: a significant proportion of the inaccuracies and oversimplifications in giving due credit to online media sources comes down to the fact that we tend to count only the last click before the conversion (that is, if we’re counting at all, and closing the loop as we should be trying to do).

Now that we agree on the fact that last-click-only attribution stinks (for a competing view, see the previous column in this series), we can begin to come to terms with some fairly straightforward confusion in spend allocation in our online campaigns. Yes, Google AdWords conversion tracking as well as Google Analytics use last click attribution. And happily, that probably won’t last forever. And yes, we should look for ways of improving on this flawed model by augmenting our model with more sources of information.

But arguably, last-click-attribution leads, at worst, to minor data discrepancies. It often isn’t fatal to the project of attempting to correctly allocate budgets. And that’s really where the challenge – with a small c – lies for most of us practitioners. We’re trying to spend and bid accurately. So we want to act in a more enlightened way. We’re looking for additional clues that will help us to spend better.

Stop the double-counting

A related issue is that we’re trying to avoid getting ripped off, trying to avoid the “overspend” problem that can dog any media that makes inflated claims about its value. Jeff Greenfield and Mark Hughes, authors of a white paper, The Hushed Hidden Gaps of Online Media Tracking, examine such serious flaws in online measurement methods. They derive in large part from the imperfect user tracking technologies we use, including tagging and cookieing methods that lead to rampant double- and over-counting. Certainly, by continuing to tell tall tales about the effectiveness of particular segments of media, without sorting out the duplication in credit, it’s fair to say that we’re falling short of where we should be.

But are some of those tales taller than others? And might the de-duping be relatively simple to achieve, say, by looking askance at claims for the influence of view-throughs from display? Or by simply bidding less on YSM, Bizrate, and Google if our bottom line sales don’t agree with the inflated attribution claims in these channels?

I hope so, because I have a problem with the next step in the buy-in towards “comprehensive” attribution models.

(And I think so, because quite a few clients of ours focus mainly or exclusively on clicks from Google, so the threat of duplicate counting from display or Bizrate, say, simply does not exist.)

Some voices in the debate—like Greenfield and Hughes—will try to woo us into making this into a Big C “Challenge”. And lead us to believe that with their model, or by using a more sophisticated model, we can get closer to the Big T “Truth”. Be suspicious of such claims, in part because the technical tweaking needed to get closer to proper attribution is going to be well within our reach at low cost in the coming year or two, included with the platforms we already know and use. We won’t have to rely on “groundbreaking” technology to tell us how to count.

(Greenfield and Hughes refer to some tracking methodologies that aren’t unheard-of in the web analytics world today, to do with recording multiple attribution sources and “holding” these in a centralized report. But their term for it – a “synthetic cookie” – is not only non-standard, it isn’t in fact mentioned by anyone else, anywhere. It appears only in their white paper PDF. To prove it, I searched that exact phrase in combination with a whole variety of industry-specific terms (such as [“synthetic cookie” analytics]; [“synthetic cookie” attribution]; [“synthetic cookie” coupled with sterne, kaushik, peterson, novo, and so forth]). Arguably then, after this article is published, the term “synthetic cookie” will have only had a single mention in any legitimate source other than their own white paper – and that’s right here. I’m all for coining useful phrases, but analytics and measurement cry out for standards, not freelancing. Especially when they’re intertwined with the user’s rights, privacy policies, privacy legislation, and of course, claims for big-T Truth and big-E Enlightenment.)

Stop the whining

The least constructive claims of all, it seems to me, are the players who believe that the whole reason we’re having this debate is so various stakeholders in a company or in the media buying industry should get “credit” for making their organization or their client money. This is the us-vs.- them style of argument that treats attribution debates neither as a scientific problem to be solved nor as a practical fact-finding mission to help in making small adjustments, but rather, a rhetorical type of exercise with all the sophistication of two six-year-olds arguing that the other kid got an outsized chunk of birthday cake. Where were they when they had the whole cake to themselves? Mighty quiet, I suspect.

In MediaPost, Dan Eggleston recently charged:

“Digital agencies generally do not take into account baseline brand awareness or brand equity or the impact of any offline media on online transactions. This is done despite the fact that studies on click-through rates for online display media show higher response when supported by a TV campaign.”

A glaring hole in Eggleston’s calculations is the actual cost of creating brand equity and buying offline media. And thus, the accompanying ROI calculation. A vague sense of higher “response” from a (relatively cheap) digital campaign supported by a (costly) TV campaign may be a start, but barely. We haven’t even reached the point of asking about sales conversions, but it’s nice we’re at least talking about response (clicks). Some actors in this debate don’t even like clicks, as I’ll review in Part 3 of this series next month.

This flavor of debate was bound to emerge at a crucial stage where traditional offline media budget costs (are we including the cost of the lavish creative?) are being questioned even by the most traditional CEO’s. In essence, the pockets of the bloated offline ad spending world that are ROI-negative can only be brought closer to break-even by stealing back “credit” from media that do work.

Eggleston claims (based on estimates from a study):  “Online media can be taking credit for as much as 15% of online conversions that were actually driven by offline marketing executions.”

All I can say is you can have credit for all of the sales if you want to take the blame for all of the spend. :)

Revenue attribution in digital media is indeed tough. The waters get muddier when the cynical enter the debate. Our t-shirt cannons are merely appropriate David-style responses to the claims of the traditional agency Goliath. Online retail represents only 6% of total US retail sales! If anything, its growth is slowing, even peaking. Attribution of offline sales (from online campaigns and online word of mouth, and all offline sources) is a much bigger attribution issue. So can we just be left alone to do our thing?

We don’t have to trust last clicks entirely, but we can use them. And we can build on them. And use a variety of methods and statistics to further inform campaign planning. No need for messy food fights. And yet, no need to back up the truck and spend $200,000 on an attribution system that claims it can help us find Ultimate Truth, either.

I’ll close with the simple words of Chris Anderson: “Google isn’t just taking more of the advertising pie—it’s also making a bigger pie.” Imagine that!