• http://www.ioninteractive.com allenkristina

    Very interesting take on A/B testing, Matt. I’m looking forward to part two in the series to learn more.

  • http://www.findmefaster.com Matt Van Wagner

    Thank you, Kristina

    Is there anything in particular you’d like to see addressed in the next column?

    Have you observed any strange phenomena such as ad group performance declining after pausing lower CTR ads?

    Maybe we’ll go to a Part 3 after I present a more objective analysis in Part 2.


  • jesse223

    I have definitely noticed the phenomena of pausing lower performing ads and seeing a subsequent decrease for the well-performing ad.

    I have wondered if this may be the result of the inaccurate data population between ads. And wondered if there really needs to be a much longer window to see a true top-performing ad. That being said, I don’t know what that time would be. Longer than 2-3 months? Depending on the amount of traffic, I guess.

    But your article makes sense in that users will see different messages on the variations. I think that is a good approach. I come accross several ads that perform within target CPA, and it is hard to make a decision. Then again, I am always worried about the fact that I could pause one ad and the adgroup will suddenly take a turn for the worse. Strange.

  • http://www.findmefaster.com Matt Van Wagner

    Hi Jesse

    Thank you for commenting. I suspect that many PPC managers have observed exactly what you are describing, couldn’t find a rational explanation for it, and just walked away from it scratching their heads.

    The tough part about your performance result is that there is no easy way to isolate the root cause. There are many suspects to implicate, including rotation of ads, length of the consideration cycle, keyword match-type /ad group interaction, seasonality and so on,

    I’ll dig into this and other issues in more depth in next months’ column. Stayed tuned…