Search Engine Land
  • SEO
    • > All SEO
    • > What Is SEO?
    • > SEO Periodic Table
    • > Google: SEO
    • > Bing SEO
    • > Google Algorithm Updates
  • PPC
    • > All PPC
    • > What is PPC?
    • > Google Ads
    • > Microsoft Ads
    • > The Periodic Tables of PPC
  • Focuses
    • > Local
    • > Commerce
    • > Shopify SEO Guide
    • > Content
    • > Email Marketing Periodic Table
    • > Social Media Marketing
    • > Analytics
    • > Search Engine Land Awards
    • > All Focuses
  • SMX
  • Webinars
  • Intelligence Reports
  • White Papers
  • About
    • > About Search Engine Land
    • > Newsletter
    • > Third Door Media
    • > Advertise

Processing...Please wait.

Search Engine Land » Channel » Analytics & Conversion » Why Big Testing Will Be Bigger Than Big Data

Why Big Testing Will Be Bigger Than Big Data

Big data is a big topic these days, one that has made its way up to the C-suite. The CMO may not yet fully understand what big data is, exactly. But the CMO knows he or she needs a plan for how to use it. In fact, three of IDC’s Top 10 predictions for CMOs […]

Scott Brinker on January 30, 2013 at 9:42 am

Big data is a big topic these days, one that has made its way up to the C-suite. The CMO may not yet fully understand what big data is, exactly. But the CMO knows he or she needs a plan for how to use it.

Big Data, Big Testing, Big Experience

In fact, three of IDC’s Top 10 predictions for CMOs in 2013 revolve around mastering this data explosion in marketing.

In many ways, this attention on big data is a breakthrough moment for marketing. Sure, data has always been present in certain corners of the marketing department — especially with you astute search marketers and conversion optimization pros. But marketing management and culture have thrived more generally around gut instincts, creative concepts, and compelling communications.

Marketing has been about big ideas. Big thinking. Big budgets.

But big data? That’s something new. What’s qualitatively different — and somewhat ironic — is that big data actually promises more visibility into ever smaller circles of customer segments, asymptotically approaching a fully personalized “segment of one.”

With the new tools that are emerging, marketers can crunch their petabytes of disparate data, pulled from owned, earned, and paid media, mingled with transaction histories, mixed with profiles from third-party exchanges, and combined with public and industry statistics to divine all kinds of interesting correlations.

The goal: uncover connections that appear to influence different subsets of your audience to take action more successfully.

The Big Hypothesis Generator

I choose the words “appear to influence” with care. Because correlation is not causation. The insights generated by big data are usually tentative at best — possible relationships, in the past, between our behaviors as marketers and the behaviors of our prospects and customers.

In other words, most of these insights are the seeds of hypotheses. There’s no guarantee that the correlations unearthed in big data can directly influence customer behavior.

There are many reasons for this. For one, while big data naturally implies a large amount data, it’s far from complete. There always remains a huge number of “confounding factors” out there in the world, not captured in our data, influencing customers in ways not represented in our closed big data models.

(In a way, this is good news for marketers: the CEO is unlikely to be able to replace you with a robot, at least, any time soon.)

But it does mean that we have work to do to harness the insights that big data nominates as possible opportunities. We should look to big data for inspiration — and combine it with our ability to distill those revelations into testable customer experiences.

That’s the powerhouse human-machine combination that lets us move from interesting correlations to actionable marketing.

Big Testing: Big Ideas, Big Team, Big Deal

However, to take advantage of this, marketing’s culture must shift towards testing.

For years, testing and optimization have been niche practices in the marketing department. A/B testing with a few direct mail pieces in the past. A/B testing with a few landing pages in the present. But, most marketing programs have been run on intuition.

But now, big data is opening the door to the executive suite for a more hybrid analytical-creative method. The questions big data raises — okay, how do we use this data to grow our business? — have an answer: broadly embrace testing and controlled experimentation as the new “operating system” of marketing.

The answer is big testing.

Big Testing

What exactly does “big testing” mean? Its bigness is a function of three things:

First, big testing is about experimenting with big ideas. This is best captured in an article by Eric Ries, author of The Lean Startup: learning is better than optimization (the local maximum problem). It actually points out that most landing page and website optimization programs, while useful in some ways, are not very helpful at learning how to build a better business.

“The right split-tests to run are ones that put big ideas to the test,” he writes. “For example, we could split-test what color to make the ‘Register Now’ button. But how much do we learn from that? Let’s say that customers prefer one color over another? Then what? Instead, how about a test where we completely change the value proposition on the landing page?”

Embracing big ideas in big testing is about fearlessly answering the question — Who’s Afraid Of The Big Bold Test? — with a resounding, “Not us!”

Second, big testing is about empowering many people in the marketing organization to do testing. It’s about giving them the training, the tools, and — most importantly — the mandate to test new ideas.

Historically, testing has been restricted to a small number of gatekeepers. But now that we have such a fragmented and fractured marketing landscape — and with big data helping us identify ever more granular opportunities within it — we need to tap more marketers on the team to run controlled experiments.

Hal Varian, the chief economist at Google, has said that Google runs about 10,000 experiments each year. A large number of different people throughout the company are engaged in all kinds of different tests in parallel. It’s not the cult of a few; it’s the culture of the many.

Finally, big testing is about making a big deal about testing from the top down, fostering a culture of experimentation.

This last point will probably be the most challenging, as culture is not something that changes quickly. Executives need to make a conscious effort to encourage real testing — starting with the acknowledgement that good experiments prove or disprove hypotheses. Not every test will be a winner, but if the test was executed well, a negative result shouldn’t reflect poorly on the tester.

Marketing leaders need to make their teams feel good — not scared — about testing those big ideas.

Big data is like fuel. Big testing will be the engine that turns it into forward momentum.


Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.


New on Search Engine Land

    Google search results spam for ‘Bill Slawski obituary’ shows the dark side of SEO

    New mobile Google ad experiment puts favicon in-line with display URL

    Google launches video health tools to help publisher monetization

    SEO pioneer and expert Bill Slawski passes away

    New Yelp feature: Request a Call

About The Author

Scott Brinker
Scott serves as the VP platform ecosystem at HubSpot. Previously, he was the co-founder and CTO of ion interactive.

Related Topics

Analytics & Conversion

Get the daily newsletter search marketers rely on.

Processing...Please wait.

See terms.

ATTEND OUR EVENTS

Learn actionable search marketing tactics that can help you drive more traffic, leads, and revenue.

March 8-9, 2022: Master Classes (virtual)

June 14-15, 2022: SMX Advanced (virtual)

November 15-16, 2022: SMX Next (virtual)

Learn More About Our SMX Events

Discover time-saving technologies and actionable tactics that can help you overcome crucial marketing challenges.

Start Discovering Now: Spring (virtual)

September 28-29, 2022: Fall (virtual)

Learn More About Our MarTech Events

Webinars

Take a Crawl, Walk, Run Approach to Multi-Channel ABM

Content Comes First: Transform Your Operations With DAM

Dominate Your Competition with Google Auction Insights and Search Intelligence

See More Webinars

Intelligence Reports

Enterprise SEO Platforms: A Marketer’s Guide

Enterprise Identity Resolution Platforms

Email Marketing Platforms: A Marketer’s Guide

Enterprise Sales Enablement Platforms: A Marketer’s Guide

Enterprise Digital Experience Platforms: A Marketer’s Guide

Enterprise Call Analytics Platforms: A Marketer’s Guide

See More Intelligence Reports

White Papers

Reputation Management For Healthcare Organizations

Unlock the App Marketing Potential of QR Codes

Realising the power of virtual events for demand generation

The Progressive Marketer’s Ultimate Events Strategy 2022 Worksheet

CMO Guide: How to Plan Smart and Pivot Fast

See More Whitepapers

Receive daily search news and analysis.

Processing...Please wait.

Topics

  • SEO
  • PPC

Our Events

  • Search Marketing Expo - SMX
  • MarTech

About

  • About Us
  • Contact
  • Privacy
  • Marketing Opportunities
  • Staff

Follow Us

  • Facebook
  • Twitter
  • LinkedIn
  • Newsletters
  • RSS
  • Youtube

© 2022 Third Door Media, Inc. All rights reserved.