A Guide To Understanding Big Testing & Massively Parallel Marketing

Last month’s column on Why Big Testing Will Be Bigger Than Big Data — encouraging marketing experimentation on a much broader scale than ever before — was well received.

But one question came up several times in the comments: how do you enable many marketers in an organization to run experiments at the same time without interfering with each other?

Massively Parallel Marketing

The idea of empowering many marketers to engage in testing at the same time — not just a small subset — is a case of what I call massively parallel marketing.

Massively parallel marketing is derived from the idea of massively parallel computing in computer science. In parallel computing, you take a large computational job, break it into smaller pieces, and let dozens or hundreds of processors work on those pieces simultaneously. The results from those individual processors are then combined into the final answer.

This is in contrast to serial computing, where one processor works on the large job all by itself, piece by piece, until it’s complete. Parallel computing is dramatically faster because you don’t have to wait for one piece to be done before you move on to the next one; you can compute all the pieces at the same time.

Massively parallel marketing applies that model to marketing, where individual marketers are analogous to processors. Just as certain jobs in computing that lend themselves nicely to parallel processing, certain kinds of work in marketing can be effectively parallelized too.

There are two requirements for work to benefit from parallel marketing:

1. The work must be able to be partitioned in a logical way, so that each “piece” can be worked on at least somewhat independently of the other pieces
2. Each piece must benefit from having a human being working on it: creativity and judgment are valuable to the work being done

Social media marketing is a great example of massively parallel marketing. Many different marketers can split up the work of responding to individual customers or engaging with individual influencers. There’s certainly coordination between them, but not so much that it prevents them from working in parallel.

The challenge in parallelizing marketing experimentation boils down to one overriding concern: you don’t want to subject an individual prospect to multiple conflicting tests at the same time in a way that would lead them to believe your organization is suffering from schizophrenia. This is the age of converged media, after all.

A lesser concern, but still a valid one, is the risk of multiple simultaneous experiments confounding each other’s results in the way they influence the prospect’s action.

Essentially, this is a variation of the “attribution” problem that has plagued marketing analytics since the dawn of time. In practice, as long as you’re not engaging in schizophrenic experiments, this effect is rarely dominant.

Let’s set aside the attribution issue for now, but we’ll address the bigger concern of schizophrenia in the specific context of testing in paid search.

Partitioning Experiments In Search Marketing

Paid search marketing, especially at the top of the funnel, is particularly well-suited to parallel experimentation.

Many brands already partition paid search using campaigns and keyword groups. Often, these represent different sets of touchpoints that lend themselves to being independently optimized — albeit with a bit of light coordination.

As illustrated above, you could split up those campaigns across two different teams. Each team would experiment with the ads and post-click experiences in their partition, seeking to optimize target metrics such as CTR, CPC, CPA, lead quality, revenue, etc.

Many experiments can be conducted in this environment without devolving into schizophrenia in the eyes of the prospects because, by their very nature, each keyword group usually implies a different “conversation.” As long as you limit your tests to messaging, presentation, and offers that apply to that conversation but don’t violate an agreed upon common identity for your business, you can safely run experiments on different conversations in parallel.

For instance, at my company, which sells software for creating and testing post-click experiences, prospects might reach us for a variety of different conversation starters: landing pages, microsites, conversion optimization, A/B testing, demand generation, content marketing, etc.

With most kinds of tests, we can experiment with the ads and post-click experiences for each of those terms independently of each other. We can try very different ideas for how to engage a visitor responding to “conversion optimization,” without worrying about what they might see if they subsequently click through on a “microsites” ad.

The only caveat is that we don’t want either experiment to violate our common identity. In our case, our common identity includes brand standards, product names, product pricing, and an underlying brand vision that will be consistent further downstream in the sales funnel.

Common identity elements can be tested — but they’re more tricky and require considerably more coordination.

Coordinating Parallel Teams

Within any one team, it’s important to have a high degree of communication and collaboration, since tests within a partition are more likely to have interaction effects.

For instance, the post-click experiences for “landing pages” and “landing page software” may very well service the same visitor in the same search session. You want as much synergy between those experiences as possible.

I recommend using agile marketing management within each team, to keep communication high and priorities flexible with sprints and daily stand-ups. One person on each team serves as the lead.

But, what about coordination among the teams?

7 Ways To Coordinate Parallel Marketing

There are many great ways to coordinate across teams. This isn’t a comprehensive list, but here are seven devices for coordination that I believe are particularly helpful in massively parallel marketing efforts.

1.  Internal chat software such as HipChat. This works best when it supports multiple rooms for different topics, on-the-fly group discussions, and a persistent history — so one can always go back to conversations later. In the context of search marketing experimentation, you might have rooms focused on personas, current offers, content pieces, etc.

2.  Lead huddles where the leads of the different teams do a daily 15-minute stand-up among themselves to keep everyone appraised of what’s happening across the different partitions. This helps to quickly uncover mutual challenges and opportunities and coordinate a common response.

3. Centralized wikis where the latest information about the shared “common identity” of the brand can be found, everything from brand standards to image and content resource libraries. (True digital asset management software can be quite helpful at scale.) The latest offers and persona definitions are here for any team member to reference.

4. Shared specialists such as graphic designers or software developers. Any one team may not need a dedicated resource with these talents. But an important side benefit of these shared resources is that they can help to cross-pollinate ideas across teams.

5. Team exchange on a regular basis, where teams swap members. This not only helps to cross-pollinate ideas across teams, it also helps the collection of teams develop a more cohesive culture across the entire massively parallel marketing effort. Consider rotating 10-20% of teams each cycle.

6. Science fair get-togethers, maybe once per months, where the different teams show off their work from the previous cycle, explain some of their rationale, share insights that they uncovered in the process, and help teach others new skills and approaches that they learned.

7. Leaderboard tracking daily, or real-time performance, of the different teams. This is intended to provide a little extra motivation with friendly competition, but more importantly, help identify those teams that are having the most impact. This becomes a fast feedback mechanism for teams to learn from each other — who’s doing what that works best?

Of course, you don’t need all of this structure to engage in marketing experimentation. You can start with a team of one.

But, if the size of your business presents the opportunity to benefit from large-scale experimentation through massively parallel marketing, there are definitely ways to operationalize that.