• http://twitter.com/soloportfolio Clare McDermott

    Hi Scott… Really interesting article. I’d love to know more about what this looks like: “Historically, testing has been restricted to a small number of
    gatekeepers. But now that we have such a fragmented and fractured
    marketing landscape — and with big data helping us identify ever more
    granular opportunities within it — we need to tap more marketers on the
    team to run controlled experiments.”

    I love the idea in concept, but some examples of how this plays out would help me understand it better.

    By the way, we are talking about Big Data in our August issue of CCO… would love to be in touch to find out how we can access your brain!

  • Pat Grady

    You need to add a “Big Mistakes” blog to that graphic. It includes report blindness, faith errors, bad assumptions, and attribution stupidity.

  • http://twitter.com/chiefmartec Scott Brinker

    I was going to, but I was afraid that might generate Big Laughs. Where would I fit that on the diagram then without creating a Big Mess?

    Seriously though, I agree with your point. All of this boils down to people and organizations being better at using data well. It’s a big change in mindset, culture, and behavior. But when I think of “big testing” that’s part of the transformation in data sensibility that is necessary to make it useful.

  • http://twitter.com/chiefmartec Scott Brinker

    Thanks, Clare.

    Happy to chat with you anytime. A couple of books that capture this phenomenon of many small tests include:

    Little Bets by Peter Sims
    Lean Startup by Eric Ries
    Uncontrolled by Jim Manzi

    Lots of good examples in those books that cross many domains (including marketing).

  • Jaume Clotet

    Hi Scott, thanks for your article, I am 100% with you. I would like to place you a question. As we can observe demand (users) are evolving faster and deeper than the offer (advertisers/companies), there is a big gap, and is getting bigger and faster everday. I see “big data” like the rope that will allow me to first stop the bull, and finally control it. Well, first of all I need to tie the beast. Now, How do you suggest put the noose around his neck, convincing the boss? Cheers!

  • http://twitter.com/TylerHakes Tyler Hakes

    I love this in concept – but I also see that it gets messy quickly.

    For one, having multiple people running multiple tests in parallel makes it hard to generate any meaningful conclusions unless you have a PERFECT data set (my experience is that almost almost no company has flawless, complete data about everything they’re doing).

    More importantly, each test that answers one question generates an infinite number of next questions — specifically if you already are working from a highly-segmented customer data set. That means that having tons of tests occurring simultaneously may just muddy the waters to the point that it becomes worthless.

    Interested to hear your thoughts on how to deal with these problems.

  • http://www.facebook.com/justin.fogarty Justin Fogarty

    Great article Scott. Testing is key if we want to use big data to create “personalized segment of one” for customers based on their intent.

    The only thing I would add is that in order to do conduct those tests, analyze the results and act on the insights at at a big data scale and at the speed needed to delight customers, machine learning will need to play a role.

    I expanded on this more on our blog today – http://www.bloomreach.com/2013/02/big-data-needs-big-testing/