3 Steps To Adopting A New Attribution Model In A Large Enterprise
When I first got to Yahoo! and asked about attribution models in our marketing team, our marketing analytics lead informed me that we had developed, and subsequently scrapped, a very elegant attribution model. In fact, we were back to square one with a last-ad model, the same one that’s been dominating our industry for what […]
When I first got to Yahoo! and asked about attribution models in our marketing team, our marketing analytics lead informed me that we had developed, and subsequently scrapped, a very elegant attribution model.
In fact, we were back to square one with a last-ad model, the same one that’s been dominating our industry for what seems like forever. When I asked why we abandoned the obviously superior system, I was surprised by the answer I got.
Hard To Sell
The attribution model developed by our analytics team was quite elegant, apparently. It accounted for ad interactions from various marketing channels, it differentiated between views (impressions) and clicks, and even weighted ad interactions differently depending on where in the conversion stack they occurred.
The real problem was that while mathematically sound, the model was simply not sellable. That is, while the analytics group (and some others) believed that the model was valid, they simply couldn’t get the various groups in the organization to buy into it.
A Cautionary Tale
I’ve since corroborated this story with countless others in the industry and it’s unfortunately a common tale, particularly in large companies. In fact, it seems that the successful adoption of complex attribution models in big organizations is more the exception than the rule.
The ones who do have a successful track record seem to be the same companies that have statistical modeling in their DNA (think credit cards, insurance, finance, etc.) and thus may be comparatively comfortable with the idea of tinkering around with complicated attribution schemes within their marketing groups.
The lesson I learned is that while it seems a significant enough task to get the attribution model right from a mathematical and statistical perspective, in big firms, there’s a whole separate set of issues around getting buy-in so that any model can actually be successfully adopted and put into play.
Given the immense challenges of developing and selling a new attribution model internally, what’s a search marketer to do?
OK, here’s what you do:
- Find a model you can sell internally
- Test it in a (somewhat) controlled environment to validate it
- Evolve your attribution model and repeat #1
Walk In A Straight Line
It may seem like a cop out, but try selling a mathematically inferior model internally as a first step. I hate to suggest this, and it certainly goes against my better judgment, but in a big corporation sometimes you just have to take one for the team.
Try a linear model that simply accounts for multiple ad events like impressions and clicks, and weighs them all equally. Think about it. Your current last-ad model completely ignores any ad interaction other than the last view or click before conversion.
Even if you’re wrong (which you are, by the way), you’ll be one step further away from your last-ad model, which means you’re one step closer to a model that actually makes sense.
Log Some Miles
If you’re feeling brave, here are some alternatives to the obviously flawed ‘linear’ model. First, try weighting views differently than clicks. How much? Try half! It’s wrong as well, but see above – it’s probably closer to reality than equal weighting. Confident still?
Try a ‘geometric’ model. This model has ad interactions gaining weight as they get closer to conversion, with the differences in weight evenly distributed throughout the curve. Simple enough to calculate, also flawed, but still fairly digestible.
Still hungry? How about a ‘logarithmic model’ that weights ad interactions exponentially more the closer to conversion they are. I’m not particularly bullish on this one, but depending on how statistically oriented your audience is, it might fly.
Test & Control
It may be necessary to validate the need for attribution before taking any model and trying to sell it internally. If this is the case (as it most often is), you should probably set up a test/control scenario where you can objectively evaluate the impact of additional ad exposure to users. There are many ways to do this depending on the type of business you have.
If you’re a publisher (like we are) it’s not too difficult to control a group of users and make sure they don’t see ads, then evaluate their behavior relative to users who do. If you’re an advertiser, you might segment users based on the number of ad exposures and validate that their behavior differs accordingly.
The unfortunate truth is that, if you keep working on developing an appropriate attribution model for your business long enough, the complexity of the model will almost certainly exceed people’s ability to aptly comprehend it. That’s why it makes sense to start by getting people used to alternatives that they can understand.
If you move slowly away from a last-ad model, people will understand the inherent tension between a model that actually works, and one they can readily understand. At that point, they will likely come to accept the reality that a good model will probably only make sense to a statistician, and not a marketer. Once this acceptance takes place, the real work can begin.
After talking to some very qualified folks in the attribution management business, I’ve come to realize that there is no standard set of models that make sense for all, or even most businesses. Every business will require a different type of attribution model, and the best we can hope for now is a consistent framework that can be applied to each business problem, where the outcome is an attribution model that works for our particular business.
I’ve written about this in the past, and won’t re-hash here, other than to say that this might be a good time to bring in outside help if you haven’t already. I still maintain that marketers (like myself) aren’t qualified to cook up attribution systems. It’s like handing me the keys to a NASCAR rig and expecting me to compete in the Brickyard 400. Not gonna happen, people.
There is one thing, however, that my experience tells me. Just like search marketing, I believe we’re all going to be engaged in attribution management sooner or later, and just like search marketing, we’ll figure it out.
Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.
New on Search Engine Land