More Hardcore A/B Testing: V2 Of The Free AdWords Script
After sharing a testing script last month, columnist Daniel Gilbert reviewed feedback and used it to make improvements.
Last month, we shared an AdWords Script to let you test anything and received countless examples of people using this creatively for PPC success. Based on your comments and feedback, we’re now sharing version 2, which is even better!
The script has added statistics functionality, emailing you immediately when your test has significant results for click-through rate (CTR) or Conversion Rate (CR). It can also cope with Shopping campaigns — which the native AdWords Campaign Experiments can’t even touch.
It still works in the same way: you make two copies of your campaign, and pause the original. (That means both tested campaigns have no history or built up quality score, which could skew the results.) One copy you label the “Control” and leave alone. The other you label as “Experiment” — this one gets changed with whatever it is you’re testing.
New Shopping Campaign Capabilities
If you want to test Shopping campaigns, it’s the same: copy a campaign twice and label the variants “Shopping Control” and “Shopping Experiment.” That wasn’t possible in the first version of the script, as AdWords Scripts treats Shopping campaigns separately from Search/Display campaigns.
Don’t worry if you’re not using Shopping campaigns. At the top of the code, just set shoppingLabelA and shoppingLabelB as blank, and the script will only look at your Search/Display campaigns.
Then, set up the script to run every hour. If you’ve not done this before, here’s how:
In AdWords, click on “Bulk operations” in the sidebar, then click “Scripts” or “Create and manage scripts” to get to the Scripts page. Hit the red “+ SCRIPT” button to create a new script, then copy and paste in our code, changing the variables at the top as required.
You’ll need to name the script (something like “A/B Testing Tool”) and click “Authorize now” so that the script can run. Then, hit “Close” so you get back to the main Scripts page, and click “+Create schedule.” Set the frequency to “hourly” and save.
There are a few variables to change at the top of the Script:
- campaignLabelA and campaignLabelB are the labels for the Search/Display campaigns you’re testing. If you’re only testing Shopping campaigns, set these to “”.
- shoppingLabelA and shoppingLabelB are for Shopping campaigns. If you’re only testing Search/Display campaigns, set these to “”.
- confidenceThreshold is used for the statistical testing. We suggest leaving it as 0.95, which means you get an email when there’s a 95% chance the campaigns are statistically different.
- reportDate is the date range for the statistical checking; if it’s LAST_30_DAYS, the statistics tool will check the performance of the campaigns for the past 30 days (not including today).
- And finally, so you can get emails about the statistics, add in your email address(es) in emailRecipients. You can have multiple emails here (separate with a comma), or you can set this blank if you don’t want any emails at all.
Once you’ve done all of the above, leave the script to do its magic. Make sure you pause and unpause the two campaigns on alternate schedules, so both get roughly equal traffic. You can see the results of the test in the Dimensions tab, under Campaign Labels.
But on top of this, the script performs a statistical two-tailed test whenever it runs: as soon as the difference in performance between your control and experiment is statistically significant, you’ll get an email to let you know. If you want to do this manually, you can always use our free statistical relevance calculator.
We love testing here at Brainlabs, so this tool is critical for our PPC success. We’ve used it to test time of day/day of week bidding for one of our clients and found it increased conversions by 9%. And, we’ve used it to test highly localized socio-economic bid modifiers, which increased conversion rate by 21%.
Please mention it in the comments if you find success, too!
NB – Once you actually have a statistically relevant result for either CTR or CR, you’ll get an email every hour, which is slightly irritating. This can be fixed by adding another Label and excluding it from the emails — we’ll tackle this in V3 if there’s significant demand. You can also modify the script to report on different metrics if they are more important to you.
Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.