How To Use ACE For Landing Page Testing With Minimal Risk

A common way of testing landing pages is to do a split A/B test. The test setup typically goes like this:

(a)  Identify the keyword/adgroup sets for which the new landing page will be tested.

(b)  Set up identical ad copies in the ad group but with destination URLs pointing to the original as well as the test landing pages.

(c)  Set up the ads to rotate evenly.

Most of the time, this setup requires you to change the account structure to run the experiment. This is seen schematically below.

Figure 1: Original Adgroup structure

Figure 2: Testing Adgroup structure

There are a few problems with this approach:

• It is not scalable: If there are M ad copies in an ad group, you have to create 2M ad copies in the new structure to do the A/B landing page test.
• You cannot control impression serving (beyond a point). By enabling an even rotation, you split traffic evenly across the ad copies. In the above example, each landing page gets 50% of the impressions.

While one can overcome the first problem with automation tools, the second issue is problematic esspecially when testing landing pages in conjunction with high traffic brand keywords.

If the new landing page doesn’t perform well, you run the risk of poor quality of conversions for half the traffic. There are many ways one can over come this, but they tend to be cumbersome to set up and also make the testing process complicated. The ideal solution, would be to:

• Not change the mix of ad copies across the impressions that the adgroup gets.
• Show a user defined percentage of impressions the new ad copy. If you are risk averse, only a small percentage of clicks should be served with the new landing page.
• Require minimal work on part of the experimenter to set up the experiment.

5 Steps To Using Adwords Campaign Experiments

The Google Adwords Campaign Experiments (ACE) platform offers a nice way to execute these landing page experiments. While detailed instructions on their setup have been provided here, I am going to skim over the mechanics in connection to the landing page experiment. The ACE platform setup has four necessary steps, and one followup action to take based on the results.

1.  Experimental setup

This can be done in the Campaign settings tab under the Advanced Settings option:

The key input here is the Control/Experiment split which enables you to decide what percentage of traffic goes to the experimental landing page vs. control. In this example, I have picked only 10% traffic to the experiment, as I am risk averse.

2. Make  the experimental change

Next, you need to decide the experimental change. In our example, we create an identical adgroup to the one we want to test, and change the destination URLs of the adcopies to the new landing page. Once the ad group is created, you have to choose the original ad group as the control and the new duplicate ad group as the experimental one. To do this, click on the empty beaker next to the adgroup column in the campaign screen.

3. Run the experiment

This can be done manually by clicking on the Settings>Advanced Settings> Experiment> chose Start Running Experiment button or automatically if the start and stop date of the experiment was selected in step 1.

4. Analyze the results

Here is an example of the type of results you would see:

Note that the test ad group got approximately 10% of all impressions which is what we wanted. You can run a test (Z test) to check if the conversion rates are different with statistical confidence.

In this case, the test does show that the conversion rates are statistically different and the test landing page is better than the existing one. Incidentally, the Adwords editor does provide statistical measures of confidence in their reporting itself and you can find details in Google’s help pages.

5. Deploy your results

You can now test the results on a larger sample (50/50 or 30/70 split) to ensure that the new landing page will perform when more traffic is diverted to it. If you feel confident however, you can divert all traffic to the new landing page by going to the settings tab of your campaign and clicking on Advanced Settings>Experiment> Apply: Launch changes fully button.

In this way, you can A/B test new landing pages on high volume keywords without risking significant downside as well as account restricting headaches.

While, ACE is an excellent and flexible platform for advertisers for many types of A/B tests, it would be great to see  more testing features such as multi-variant testing capabilities in the future.

I would like to thank Rob Levetsky, Sr. Account Manager at Efficient Frontier for providing me with several examples and use cases of Landing Page testing with ACE.