• http://www.rimmkaufman.com/ George Michie

    Terrific piece, Sid. I was about to raise the time duration issue (lifts impressions by X% for how long?) but then you raised it yourself in #4.

    Everyone in our space is interested in proving the offline impact of online ad buying, but too few folks are willing to acknowledge much less study the impact of offline marketing online. Kudos for doing so!

  • http://twitter.com/serbayarda Serbay Arda Ayzit

    This is really nice topic, do you count only ppc part in your model? And you didn’t take search clicks to the web site, is there any reason not to take it? 
    One more thing, i think we must take organic searches in this model too but not provided issue will destroy data quality am i right?

  • Armen Avedissian

    Excellent article indeed, this is a good starting point, however there are so many other areas to take into consideration, would enjoy discussing more in detail.

    At Convertro we specialize in exactly these areas and measure the impact of offline on other offline channels as well as all online channels.




  • http://www.facebook.com/profile.php?id=15907244 Kenny Red Leader Schmied

    A great place to start & I’m glad you were forthright with the limitations of such a simple model. I briefly modelled media mix in one of my previous positions, and we used GRPs rather than ‘spend’ on advertising as our independent var from the TV channel. Since GRPs can vary in price depending on the sub-channel, it may be more advantageous in such a simple case to model reach vs. impressions rather than spend vs. impressions – you may get a more consistent, and actionable result if you invest in more than one TV sub-channel.

    I am also glad you brought up the lag issue, as it’s definitely a big one when trying to model any marketing mix, especially any type of online-offline attribution. Remember, the lag will also depend on the type of advertising and the channel itself. TV may leave a longer lasting impression than radio, or a branded special promotion might have more staying power to fuel searches than a typical, run-of-the-mill awareness advertisement. That being said, I would caution all but the most learned and experienced in econometrics from attempting ARMA or ARIMA modeling. A simple AR model – even for media mix – should do the trick in a simpler way without much risk of leaving out any important indicators. You can use the partial autocorrelation function to help determine your starting lag lengths.

  • http://www.facebook.com/people/Sid-Shah/100002282332518 Sid Shah

    George: Thanks for the compliments. I had a tough time writing this piece because I wanted to keep it simple but not over simply it.

    Serbay: Yes, you can measure the impact on total clicks too. The dependant variable will change and I am sure you will see organic clicks jump too. For this dataset I did not have organic or direct load data.

    Kenny: You bring up a good point about GRP. I have done simple modeling of GRP vs SEM clicks and TV spend vs clicks and found GRP to have a better connection to SEM clicks.

    I am curious to know why you caution against ARMA models. Could you explain ?

  • http://www.facebook.com/profile.php?id=15907244 Kenny Red Leader Schmied

    Hi Sid,

    It’s my personal process to try and use the simplest form of modeling I can to achieve a certain goal, while still preserving data and statistical integrity. When you move from the simpler models, like ols, gls and ar, to the more complex models, such as arma, arima, arch, mle, or others etc. There are many more ‘quirks’ and ‘rules’ the data must follow for the results of these more complex models to have statistical validity and be actionable. For instance, in ARIMA modeling with time series, you have to start concerning yourself with random walk, or whether or not the model should contain a constant, among other things. 

    Even having taken 4 graduate econometrics courses myself, I find I am hesitant to explore these more complicated models in an applied setting. Among other things, I have time constraints to meet. If an ARIMA model would be 5% more accurate, but take 40% more time to construct, is that an acceptable margin of error for the project I’m working on? Most of the time – yes. 

    As for why others should possibly avoid the more complicated models, most people who have taken a statistics course are familiar enough with OLS to know about heterskedasticity and the other ills associated with linear regression – at least enough to where their results will be, in general, actionable and statistically valid. It’s kind of hard to really mess up OLS. But with the more complex models, they’re likely teaching themselves along the way, or possibly just plugging numbers in to MiniTab and hitting ‘go’ without any proper knowledge of whether or not their results even mean anything!

    I’m certainly not trying to steer people away from learning these more complex statistical models. But it does take time. They certainly have their place, but I find it lies more with the academic world than the (typical) fast-paced business world. 

    This is a thrilling discussion, to be sure, and much kudos for bringing the more advanced side of econometrics to light. There is a much, much deeper hole than OLS, afterall :)


  • PeckLorrie254

    Here are few tips by using which you can earn so much money in couple of days. In start you will face so many difficulties but later on you will get various ways to earn money over Internet. Everything takes some time to get to know, same the case with it, but we will tell you the kick-start way of earning through Internet which would be much more worthwhile for you. Last week I got a site, where you can kick a good start, Look this for further details.  http://earnusdathome.weebly.com/

  • http://www.facebook.com/people/Sid-Shah/100002282332518 Sid Shah

    I quite agree with your philosophy. I like to keep it as simple as possible if it gets me far enough to the answer. There is an opportunity cost of time in business !

    As far as AR/MA/ARMA models.. I have always had a bit of a philosophical issue with them. I like deterministic models to explain trends and only use time series models when I dont know the mechanics of the underlying process.

    Anyway great discussion !