Creative Testing For The Advanced Search Marketer, Part 3

For paid search programs, big and small, creative optimization remains one of the single most impactful strategies for increasing traffic, lowering costs and acquiring more revenue. Continuously generating, analyzing and iterating on new creative delivers incremental improvements in keyword-to-creative relevancy.

Increasing relevancy results in higher click-through-rates (CTR) and Quality Score, and as a result, lower costs. Furthermore, testing helps marketers discover more compelling creative messaging to increase visitor engagement with the conversion funnel after the click, promoting higher conversion rates and more revenue.

Last month, we looked at leveraging dynamic keyword insertion, prioritizing tests based on return and limiting the number of test elements. Today, in the final installment of this three part series, we’ll review three additional best practices for conducting a successful creative test.

Check Campaign Rotation Settings

Ensuring even creative rotation is one of the most commonly overlooked steps when implementing a creative test. Google offers three creative rotation settings: optimize for clicks, optimize for conversions and rotate evenly. Optimizing for clicks is the default setting, but does not promote a fair test since creative that provide more clicks, are displayed more often.

Opting to rotate evenly results in a fair and more statistically significant test, as each creative receives an even number of impressions. This setting also favors the utilization of key performance indicators (KPI) outside of clicks and conversions, such as conversions per impression or return on ad spend (ROAS), to determine top performing creative.

AdWords Ad Rotation Settings

Keep in mind that Google campaigns set to rotate creative evenly will only do so for 90 days after the last creative was enabled or edited (unless opted-out of this functionality). After this 90-day period, creative will automatically optimize for clicks. This shift in rotation setting occurs regardless of whether or not statistical significance has been reached within the test.

Focusing on higher volume groups will help to reach significance within the rotation period. Pausing poor performing creative and generating new creative for testing will reset this 90-day clock.

Implement Tracking Before Each Test

With larger scale accounts, locating old creative tests can feel like searching for a needle in a haystack. Once a new test has been launched, it’s critical to take note and track the where and when of each new creative.

Where in the account is the creative being tested? And when was the creative activated?

Marin Enterprise Test Tracking


Reaching statistical significance in a creative test can take days, weeks and sometimes months. Even for high volume groups that reach significance faster, knowing when a new creative was added defines the date range used for reporting and analysis.

In addition, low-volume groups that utilize longer date ranges are more susceptible to Google’s 90-day rotation functionality and thus require more attention. Knowing when the 90-day rotation period is nearing its end means tracking when a creative test started.

Reach Statistical Significance & Stop

Online marketers often end creative tests too early or let them run for too long. Determining a winning creative through statistical significance requires patience, and is one of the most difficult steps in a successful creative test. Achieving statistical significance leaves little doubt that a new creative outperforms the other creative in the group.

Prior to calculating significance, a KPI such as conversions per impression, CTR, conversion rate or ROAS should be selected for evaluating the performance of each creative. By definition, creative performance is statistically significant if it’s unlikely to have occurred by chance.

To calculate this, many creative testing tools utilize a Student’s T-Test with a user-defined confidence level. This test determines the likelihood that the difference between a single creative and the average of all creative in the group, has not occurred by chance.

A confidence level between 80% and 99% is standard, but keep in mind that low-volume groups do not support high-confidence levels and are not likely to achieve statistical significance as fast as high-volume groups.

Rinse & Repeat

Search marketers are constantly searching for ways to find and engage their target audience. No optimization strategy is more central to accomplishing this goal than creative testing. Continuously testing to find more relevant and more compelling creative serves to not only increase CTR and Quality Score, but decrease costs and drive more revenue.

When implementing creative tests, search marketers must remain disciplined at every step. New creative must remain focused and relevant. Tests must be prioritized, tracked and statistically significant. Adhering to best practices and avoiding common pitfalls will help ensure that new iterations of creative will incrementally improve account performance.

Though search marketers cannot guarantee that all creative tests will be successful, they can guarantee that all creative tests have been set up for success.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: SEM | Enterprise SEM


About The Author: was the Vice-President of Marketing at Marin Software, a leading platform for digital advertising management, and held a variety of executive roles at Coremetrics, one of the early innovators in web analytics.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Chris Zaharias

    Ad copy optimization doesn’t get the focus it deserves, despite being one of the three primary ways to succeed in SEM (bidding/campaign mgmt & landing page optmization being the other two IMHO). Good work bringing more attention to this area!

  • Thib NI

    It is true that ad copy optimization is often overlooked or put back in the waiting list.However, I have found that it is now almost impossible to properly test ad copy in Google Adwords. 
    In most cases, even if the campaign settings are properly set to “rotate evenly”, one BIG metric does not rotate evenly : the average ad position. It can vary greatly between different ad copies, and renders an analysis of CTR quite useless.

    I would actually love to see a post about this on SEL. Anyone ? :)

  • Carnegie Crow

    I don’t see any “advanced” search marketing here. All I read was “make sure you test properly”.


Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide