Why Most Attribution Analysis Is Fatally Flawed

“Last-click attribution is dead!

“Media-mix modeling is the key to marketing success!”

“Without an accurate attribution model you’re throwing away your marketing dollars!!”

At SMX West last week the halls were echoing with passionate cries about attribution analysis. It seemed as if all topics (other than the Yahoo-Microsoft search deal) had taken a back seat for a moment, and suddenly the most important thing to consider was attribution analysis, specifically whether or not you are giving too much credit to SEM and not enough to other media.

Believe me, I share people’s enthusiasm on the topic. It’s clearly one of the next big online marketing problems to solve. And despite the fact that moving away from last-click attribution toward a more elegant and accurate attribution model can really only serve to divert budget away from SEM and toward other channels, I do think it’s the right thing to do. But after talking to as many people as I could, gathering my own data and soliciting opinions, I’m convinced that we are a still a long way from being any good at this at all.

Currently there appear to be two basic types of approaches, both of which seem to me to be fatally flawed.

One approach I see in the market, offered by various otherwise credible services, has the advertiser entering percentages into boxes on a screen, assigning portions of the conversion value to different marketing channels—25% for SEM, 35% for display, 15% for email, and so on. As a large advertiser myself, I can safely say that this approach gives someone like me entirely too much credit as a sophisticated marketer. I don’t know anyone who has a good enough grasp on their business and the implications of attribution analysis to make an intelligent decision in this type of situation. No knock on my fellow advertisers, but seriously, this is way out of our league. Even so, a Google representative stated during a panel I was moderating, that they intend only to provide attribution-related data, placing the burden of analysis on the advertiser.

The other approach I see emerging is a black-box math-based approach. This is more likely to be done in-house by large advertisers, using statistical and predictive modeling to simulate different attribution models, and mapping their outcomes to business metrics like profit, revenue or ROI. While I do think there is significant value in doing the hard math and understanding these problems from a statistical point of view, this methodology tends to be short-sighted. I don’t believe there is a one-size-fits-all approach to attribution analysis where you simply dump your marketing data in, and out magically pops an attribution model that maximizes profit, for example. It’s just not that generic of a problem.

It’s easy for me to sit back and criticize the status quo—so why not offer some solutions, you say? Well, here goes: I envision a three-phased approach that takes some elements of the existing practices, then combines and expands upon them to provide a more complete, appropriate solution for each advertiser.

The first phase involves smart people talking to each other. Revolutionary, no? We need an attribution specialist to lead off this effort by conducting a fairly exhaustive analysis of the advertisers’ business and online marketing programs. Starting with business goals and product adoption cycle, to conversion window analysis, on to a channel-by-channel audit of both online and offline marketing. The purpose of this consulting and analysis is to provide the proper inputs into phase two.

Phase two is the super-math modeling I describe above. With the proper inputs as they relate to an advertiser’s business and its metrics, statistical modeling is needed to predict all possible outcomes and understand which model will best support the advertiser’s business goals.

Finally, phase three makes all of this actionable. We need a way to pluck the wisdom out of phase two and apply it directly to actual media channels the advertiser is running. Ideally we’ll find a way to automate this or at least automate the recommendations, which can then by easily implemented into the media buys themselves.

But before any of us sprint into the world of attribution analysis and media mix modeling, let’s step back and take a long look in the mirror: I don’t know of a way to realistically pull any of this off if an advertiser doesn’t have a common tracking/analytics system for all marketing channels. So before we start hiring expensive analysts, consultants and statisticians, let’s be sure to clean our own houses and get our own data in order. Standardize your analytics and measurement on a single platform so you can compare “apples to apples.” Then you can start to focus on the fun stuff.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: SEO | Industrial Strength

Sponsored


About The Author: is Vice President, Marketing at Move, Inc., parent company of Realtor.com and other significant real estate-focused web properties. In this capacity, Roth oversees Paid and Organic Search, Affiliate, Mobile and Social Marketing for the Company. Prior to his arrival at Move, Dave was Sr. Director of Search and Affiliate Marketing at Yahoo!, Inc.

Connect with the author via: Email | Twitter | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://www.rimmkaufman.com George Michie

    Hi David,

    Your point is well-taken. We’ve just launched our own attribution management service, and recognize both the importance of having a system flexible enough for each different advertiser and with enough analytic power to give meaningful insights.

    Because so many advertising channels are behavior driven, particularly affiliates and email, our mathematicians internal and sprinkled around University research labs aren’t convinced that fancy black box models will actually serve the need. Modeling may be unable to determine what fraction of folks who search for “Acme Coupon” after being driven to Acme’s site by SEM efforts would buy in the absence of that affiliate ad. Absent controlled tests this behavior, and digging up email offers from the inbox may make the models less accurate than good old marketing intuition.

    We’ll see, our statisticians are still hammering on it, but it may be more than a difficult problem.

  • http://www.doyoualoha.com mkematt

    Attribution Analysis (we call it “Attribution Tracking”) will be an important tool once it’s out of its infancy stage of development. The bottom line… it’s all about Marketing Mix. All marketing channels need to be “optimized”. That means developing a strategy where each channel is in alignment with the others (branding, messaging, etc.).

    What we typically see through our “Attribution Analysis” is that most conversions begin and end with Search. Regardless, we understand that a comprehensive Marketing Mix is key in driving revenue. Our efforts are spent ensuring that all of our marketing channels flow together. If one is off balance, the rest do not perform as well as they could.

  • http://www.marketingfinanceobama.com Jamie Voorhies

    I agree that current modeling is flawed, though it seems everyone’s thinking is aligned on that. As in, we all know it is broken, but fixing it is really hard!

    I’ve seen some excited stuff involving placebo tests (i.e., running multichannel DOE testing where you serve up fake ads to monitor the true lift of the varying placements). You could couple this with on/off switches on mass-market campaigns to get even greater understanding. Of course, this isn’t going to give us the answer, but just another step in that direction.

  • http://www.marketing-jive.com Jody N

    Great article David. A key piece to what you stated late in your post: I don’t know of a way to realistically pull any of this off if an advertiser doesn’t have a common tracking/analytics system for all marketing channels.

    Attribution is about giving credit where credit is due, without consistent tracking, true/accurate attribution is impossible to achieve.

 

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide