Why Most Attribution Analysis Is Fatally Flawed

“Last-click attribution is dead!

“Media-mix modeling is the key to marketing success!”

“Without an accurate attribution model you’re throwing away your marketing dollars!!”

At SMX West last week the halls were echoing with passionate cries about attribution analysis. It seemed as if all topics (other than the Yahoo-Microsoft search deal) had taken a back seat for a moment, and suddenly the most important thing to consider was attribution analysis, specifically whether or not you are giving too much credit to SEM and not enough to other media.

Believe me, I share people’s enthusiasm on the topic. It’s clearly one of the next big online marketing problems to solve. And despite the fact that moving away from last-click attribution toward a more elegant and accurate attribution model can really only serve to divert budget away from SEM and toward other channels, I do think it’s the right thing to do. But after talking to as many people as I could, gathering my own data and soliciting opinions, I’m convinced that we are a still a long way from being any good at this at all.

Currently there appear to be two basic types of approaches, both of which seem to me to be fatally flawed.

One approach I see in the market, offered by various otherwise credible services, has the advertiser entering percentages into boxes on a screen, assigning portions of the conversion value to different marketing channels—25% for SEM, 35% for display, 15% for email, and so on. As a large advertiser myself, I can safely say that this approach gives someone like me entirely too much credit as a sophisticated marketer. I don’t know anyone who has a good enough grasp on their business and the implications of attribution analysis to make an intelligent decision in this type of situation. No knock on my fellow advertisers, but seriously, this is way out of our league. Even so, a Google representative stated during a panel I was moderating, that they intend only to provide attribution-related data, placing the burden of analysis on the advertiser.

The other approach I see emerging is a black-box math-based approach. This is more likely to be done in-house by large advertisers, using statistical and predictive modeling to simulate different attribution models, and mapping their outcomes to business metrics like profit, revenue or ROI. While I do think there is significant value in doing the hard math and understanding these problems from a statistical point of view, this methodology tends to be short-sighted. I don’t believe there is a one-size-fits-all approach to attribution analysis where you simply dump your marketing data in, and out magically pops an attribution model that maximizes profit, for example. It’s just not that generic of a problem.

It’s easy for me to sit back and criticize the status quo—so why not offer some solutions, you say? Well, here goes: I envision a three-phased approach that takes some elements of the existing practices, then combines and expands upon them to provide a more complete, appropriate solution for each advertiser.

The first phase involves smart people talking to each other. Revolutionary, no? We need an attribution specialist to lead off this effort by conducting a fairly exhaustive analysis of the advertisers’ business and online marketing programs. Starting with business goals and product adoption cycle, to conversion window analysis, on to a channel-by-channel audit of both online and offline marketing. The purpose of this consulting and analysis is to provide the proper inputs into phase two.

Phase two is the super-math modeling I describe above. With the proper inputs as they relate to an advertiser’s business and its metrics, statistical modeling is needed to predict all possible outcomes and understand which model will best support the advertiser’s business goals.

Finally, phase three makes all of this actionable. We need a way to pluck the wisdom out of phase two and apply it directly to actual media channels the advertiser is running. Ideally we’ll find a way to automate this or at least automate the recommendations, which can then by easily implemented into the media buys themselves.

But before any of us sprint into the world of attribution analysis and media mix modeling, let’s step back and take a long look in the mirror: I don’t know of a way to realistically pull any of this off if an advertiser doesn’t have a common tracking/analytics system for all marketing channels. So before we start hiring expensive analysts, consultants and statisticians, let’s be sure to clean our own houses and get our own data in order. Standardize your analytics and measurement on a single platform so you can compare “apples to apples.” Then you can start to focus on the fun stuff.