Six-to-One Impact in Offline Stores

Based on two years of research conducted by retail marketing firm RevTrax, the study discovered that “for every $1 of e-commerce revenue generated from paid search, marketers can expect to see approximately another $6 of in-store revenue.”

In other words, paid search has 6:1 impact on offline sales over e-commerce. Because of the challenges of tracking consumer behavior online to offline, most of this has been invisible to marketers. Only now with the rise of smartphones and other methodologies is online-to-offline tracking becoming more widely available.

Between August 2009 and August 2011 RevTrax monitored millions of paid-search ads and consequent sales for its retail clients. To track in-store sales accurately RevTrax used landing pages with coupons and unique IDs:

• A paid search ad was displayed to a consumer
• The paid search ad led the consumer to a printable or mobile landing page displaying a coupon with a unique barcode
• The consumer redeemed the coupon inside a brick & mortar store
• Each coupon was tracked back to the online search (and the keyword)

Average Paid Click Worth $15 in Store

Using this methodology, RevTrax could conclusively determine in-store sales affected by paid search ads. Here’s how RevTrax’s findings illuminated the “value of a click” (where the average transaction size was under $200):

• The average click on a paid search ad generated approximately $15 of in-store revenue, with some merchants seeing as much as $28 of in-store revenue.
• Approximately 9% of clicks on a paid search ad generated an in-store sale, with some merchants seeing up to 26% of clicks on a paid search ad generating an in-store sale.

Again, what the company found was that paid search drove $6 in offline sales for every $1 in sales online. RevTrax thus argues that multichannel merchants who do not include in-store sales into the ROI calculation are potentially “undervaluing the paid search channel by as much as 85 percent.”

This is the first study to conclusively show the offline impact of paid search at this kind of scale, based on actual behavior rather than consumer surveys and self-reported data. The findings are pretty radical, with broad implications for search marketers and the industry as a whole.

Postscript: Here’s a bit more color and explanation from Seth Sarelson, COO of RevTrax, in response to a couple of questions I received about the study:

Clients are measuring these paid search campaigns at the keyword level and looking at brand keywords, competitive keywords, categories/products, etc. and the study uses a mix all these different types of executions to come to these results. I’m not sure that it’s clear to the person commenting that we’re talking about paid search only, not organic, so there’s certainly no guarantee that a brand is at the top of the paid search results for any keywords in a particular category.

One of the things that the study also mentioned is that many clients are reporting that 40-50% of customers acquired via paid search were new. This is big as it shows that this isn’t just an example of existing customers searching on branded terms looking for a deal.

It would certainly make sense for us to do a follow up that compares brand vs non-brand to address this specifically, as it’s an important point of distinction. I’m going to push for this internally.

As for PC vs Mobile, we’re working on a later study that will address these results, but most of what we’re looking at here is printable coupons from a PC.

Stock image from Shutterstock, used under license.

If you have the time, budget and manpower to utilize a design team and software that builds a landing page for every paid, organic, social and email marketing keyword or blast you send — Awesome!

The reality is, most of us do have time and budget limitations. Getting permission to spend thousands on software isn’t likely, but that doesn’t mean you have to give up on the idea. Think a bit deeper and build your landing pages for multiple purposes.

In the past, we would build a website, and send clicks to the homepage of that site. Then we got smart, and sent those same clicks to more relevant pages on the site. Then Landing Page Optimization was invented and we started hearing we needed a new page for every keyword, and the want vs. budget debate started heating up.

Limited budgets don’t mean you have to completely remove LPO from your marketing vocabulary. Think about it this way: if you can repurpose that page to give you Organic, Paid, Social and Email Marketing traffic and conversions, it makes the time investment a bit more manageable.

Then think about the reality that not every keyword needs a landing page. Many keywords are extremely similar, the same words in different combinations. Take this knowledge and repurpose your landing pages across multiple platforms.

Here are five steps to making sure your landing pages can be repurposed for multiple funnels.

Layout Your Landing Page

There are quite a few schools of thought about how to frame out a landing page. Some blueprints say a large image and a call to action, some do include text. The best advice I can give here is just to simply start somewhere and test what you’ve done.

A great tool to help you mock up pages as you go, and save layouts you’ve done or want to try, is GoMockingbird.com. I use this to wireframe pages for SEO, PPC, Email Marketing, Facebook inclusion, etc. It’s also a great way to show your design/creative department your vision when it comes to page layout. If you know a UX expert, or have someone on your design staff that “gets it” be sure to use their knowledge and experience.

Target Keywords

This tactic relies upon you having a very solid AdWords campaign. We’re going to use your AdGroup themes to set up your landing pages. AdGroups generally have very similar “like” terms and you’re testing ad copy and calls to action to see what converts best. Export your AdWords keywords and sort by AdGroup. Use this information to decide what landing pages you need to build for each set of terms. You can then use this information to build your Landing Page.

Add Content

After you export your keywords by AdGroup, make a list of the landing page content you need. Each “Group” of keywords needs between 150 and  200 relevant words that are themed around those keywords. Do not stuff each keyword into your content verbatim, use synonyms, write naturally – bold your main idea and don’t be afraid to use bullets to list the benefits of your product(s).

Create Calls to Action

Review your PPC ads and use the best performing ad headline as your call to action: “Book your Getaway Today” or “Limited Time Offer”. You can combine on-page testing via Website Optimizer with some PPC ad rotation testing to find the best combination of call to action and keyword group.

Track It

Sending multiple clicks to the same landing pages can make your tracking a bit tricky, but if you’re careful, you can see each portion of revenue a segment delivers. You can use a few tactics, Google offers different Analytics options and filters to help you out.

The best way, in my opinion, is to using source codes on the links and building custom segments in your Analytics platform. 

Remember, if you want these pages to rank organically as well, you’ll need to fit it into a logical place in your navigation, add content, build internal links and tie its existence in with the rest of your website. These landing pages aren’t meant to be an “add on” with no place to live in the architecture of your site.

Landing pages are designed to convert traffic, but they don’t need to be a one-hit wonder. You can combine Organic, Paid and Email Marketing tactics with your landing pages to get great results on a shoestring budget.

Adchemy says that it can “dynamically generate tens of thousands of paid-search ads directly related to intent of individuals, each with a customized landing page.” The idea is that when someone inputs “best inexpensive laptop, San Leandro California,” rather than a more “generic” ad designed to cover a wide range of keywords, Adchemy will create ad copy that contains comparable language and better match the intent being expressed in that query.

The company generates ad copy and landing pages “automatically,” based on templates, but not contemporaneously with the query. It’s not a version of the old “Find Greg Sterling on eBay” scenario. The ad copy and landing pages are created ahead of time based on keyword and query log analysis.

Adchemy SVP Anurag Wadehra argued to me that creating ad copy and landing pages manually cannot hope to capture the search intent expressed in the huge keyword lists now used by large scale search marketers. He added that Adchemy also simplifies complex search campaigns by shifting campaign management from “millions of keywords” to mere hundreds of “intent groupings.”

Here’s how the company explains the process:

• The Adchemy WordMap analyzes consumer search queries, as well as your keyword portfolio, breaking down keywords and query patterns into consistent terms and intents.
• The WordMap application then groups similar intents into Topics, such as “Brand,” “Family,” “Segment,” or “Value.”
• The WordMap application analyzes your keyword portfolio and query logs (if available) and derives Keyword Expressions — patterns of how your customers are expressing their intents. These Keyword Expressions can then be used as templates to automatically generate additional relevant keywords at scale, with each generated keyword retaining its underlying intent(s).

I haven’t seen the platform in action. But if the company can actually deliver as promised it would be impressive. Validating its claims, to a considerable degree, are the fact that Adchemy has received $60 million in funding and has 300 customers.

Search marketing platform Kenshoo earlier this summer introduced what it calls Call Conversion Optimization. The program, according to Kenshoo, automatically adjusts bidding in response to ads that are driving calls rather than resulting in just clicks. Unique phone numbers are used, at a campaign or keyword level, to determine which ads are working from a calls standpoint.

Today Kenshoo further refined the program with partner Mongoose Metrics. Now unique phone numbers can be generated for each keyword — and individual user — so that no two people searching on the same keyword will see the same phone number on the resulting landing page. According to the press release:

The tight API integration of Mongoose Metrics’ call tracking and speech recognition technology combined with KENSHOO Local’s sophisticated bid optimization algorithms, known as Call Conversion OptimizationTM (CCO), allows advertisers to leverage Dynamic Number Replacement (DNR) to display a unique tracking telephone number to each user, creating a link between campaigns, keywords, and the converting call. Unlike other solutions, KENSHOO’s CCO tracks these conversions and suggests optimal bids for the keywords that generate phone conversions and suggests reduced bids or removal of keywords that generate clicks – and cost – but do not result in the desired conversions.

Kenshoo says Mongoose is going to add call recording for additional capabilities (e.g., keyword research) in the near future.

Yext, which describes itself as a “pay per action” platform (using calls) uses speech-to-text data mining to filter good phone leads from weak ones and says it only charges clients for good calls accordingly. This is an advancement over call-based advertising that relies solely on call duration for advertiser billing.

Marchex, which operates a pay per call exchange, has bet its business increasingly on phone-based advertising and call analytics. The company last week introduced “call mining,” which takes recorded calls and speech-to-text transcription and makes all sorts of interesting analytics options available on top of that via a dashboard.

According to the company’s promotional materials the following are the kinds of things that can be done or determined:

• On-the-phone conversion rates and caller intent
• Identify the channels, ads and keywords that drive the highest on-the-call conversion and return on investment
• Map call outcomes to parameters such as geographic location and gender
• Understand the effectiveness of call-based marketing campaigns
• Audit call center efficiency and caller experience
• Directly ascertain customer needs and pain points
• Flag calls by topic of discussion

Beyond identifying which keywords generated more calls, these capabilities reveal the quality of the specific calls and enormous amounts of additional information that can be factored back into marketing campaigns (as well as operations more broadly). Marchex EVP Matthew Berk and I discussed scenarios where national advertisers might adjust campaigns (ad copy, keywords) on a regional basis according to data revealed by the content of successful phone conversions.

At a higher level these tools point the way toward the convergence of marketing and customer service, which have historically been distinct silos within organizations.

Call measurement firm Telmetrics told me yesterday that the company has seen “exponential growth” in demand from online marketers as they realize the analytics and data-mining benefits of tracking and monitoring calls.

I’d like to hear whether people believe the SEO concerns remain; however Telmetrics said it has figured out a way “through automated integration and data reconciling” to prevent tracking numbers from appearing as primary local business lines.

The larger point is that these optimization and call mining capabilities offer marketers new levels of insight into the effectiveness of campaigns that were just not possible with clicks and conventional web analytics in the past. I also believe — and I’m totally speculating here — that Google will add call analytics to AdWords (either on its own or through an acquisition) in the next six months. We’ll see.

To kick things off though, I’m going to debunk a common misunderstanding about doing things “scientifically” and reveal why marketers have a far greater advantage in this arena than you might realize at first.

The scientific method

Back in high school, you probably learned about the “scientific method.” It’s the systematic approach you’re supposed to take when conducting experiments:

1. Form a hypothesis
2. Figure out how to test the hypothesis
3. Run the experiment and collect the data
4. Analyze the data
5. Interpret the results and draw conclusions

Your conclusions in the step—whether positive or negative—in turn inspire new hypotheses, and a virtuous cycle of discovery begins.

Because these steps follow a fixed order and emphasize a rigorous approach to collecting and analyzing data, most people think of the scientific method as an analytical discipline.

Perhaps you’ve heard kids advised, “oh, if you’re analytical, you should go into math or science.” The implication is that if you’re more creative, then you should probably avoid math and science and go into something else, like marketing.

What a terrible mistruth—one that has probably cost us generations of brilliant, creative scientists (and given more than a few marketers an unnecessary complex about analytics).

Great science springs from creativity

Here’s the big secret: while the analytical aspects of the scientific method are what’s often extolled, the first two steps are driven by right-brained creativity as much—if not more—than by left-brained reasoning.

How do you come up with a hypothesis? Creative thinking. There’s no step-by-step recipe you can blindly follow to develop a great hypothesis. You have to reach into your imagination and envision a question and a possible explanation.

How do you figure out the best way to test that hypothesis? Creative thinking again. Sure, some hypotheses will have an obvious way to be tested. But many will be more challenging. Selecting the right subjects for your experiment and carefully arranging the test so that the results are not contaminated by a jumble of confounding variables is an art unto itself. Coming up with a new way to run a certain type of experiments can be an even greater discovery than your initial hypothesis, as it may enable a whole new set of possibilities to be explored.

These first two steps are where brilliance shines. Everything else is pretty much paint-by-numbers.

Without creative thinking—out-of-the-box ideas, bold imagination, lateral associations and daring originality—the analytical machinery of the scientific method never even gets off the ground. You can mechanically run experiments all day long, but without that critical injection of innovation at the top, it’s all sound and fury signifying nothing. The wheel is spinning, but the hamster’s dead.

The marketer’s advantage

You can learn a process more easily than you can learn how to be creative. Almost anyone can follow a recipe to bake decent cookies; becoming a wondrously inventive pastry chef is another matter entirely.

In this way, most marketers have a tremendous leg up in adopting scientific methods. Marketing careers are typically built in roles where creativity is encouraged, if not outright demanded. It’s embedded in our educational programs, popular books, social norms, industry awards and self-identities. And like any skill or talent, with practice and encouragement, it blossoms and improves.

As a result, marketers are some of the most creative people in business.

The irony has been that in the growth of marketing analytics and scientific marketing, many marketers and marketing organizations have artificially dampened their creativity to try to be more analytical or more scientific—as if you had to trade one for the other.

In fact, marketers should do just the opposite: wield analytics and the scientific method to push their creativity further than they ever dared before.

Make no mistake, the analytical and experiment-driven dimensions of “the new marketing” are here to stay and are now integral to success. You’ve got to be able to do the math, reason deductively and implement systematic marketing methods that feed the engine of analytics.

But to a certain degree, much of that is like following a recipe. Maybe more of a Julia Child’s The Art of French Cooking recipe than a Betty Crocker instant cookie mix recipe. But still a recipe: methods are, after all, simply methodical.

What will make you a star chef—or search marketer—will be the soul-level combination of those analytical methods with your inspired creative spirit. Make analytics the servant of extraordinary marketing, not the other way around. Embrace these new left-brain skills without surrendering your right-brain magic. (If you haven’t yet, go out and read Daniel Pink’s A Whole New Mind.)

Infuse your imagination into the science, as much as the other way around.

Truth be told, that is what the greatest scientists have always done. Who knows? If Einstein were alive today, he just might be a search marketer. Or at least working for Google.

This is called predictive heatmapping. So how does it work and what good is it to you? The answers to that plus some real case studies, after the jump.

What is an attention heatmap?

An attention heatmap is a combination of two elements: eye-gazing, and predicted attention.

Eye-gazing simulates the sequence of extremely rapid and involuntary eye movements (“saccades”) that happen as your eye scans a scene. This is overlaid on a heatmap of the attention represented by different colors which predicts where the brain will focus. Hotter areas indicate a more intense focus, while cooler areas show a lower level of awareness and importance.

What factors are considered in generating a predictive heatmap?

As Feng-GUI’s neat “how we do it” page explains, their algorithms are based on recurring themes detected via eye-tracking. These are things like color contrast, orientation of an object, etc.

How can it be used?

The description continues:

“It can be used to refine landing page designs for existing or new pages, and to improve conversion rates. It can identify which page elements are being looked at and which are being ignored. This allows the designer to focus attention on the correct parts of the page and increase the likelihood of conversion. Attention heat maps can be created several times during the design or re-design process to ensure that the refinements are having the intended effect.

“A ‘busy’ eye-gaze path and scattered heatmap with many hotspots is usually an indication that visual priorities of the page are not clear, and will likely result in confusion and a higher bounce rate for your landing page. Relatively simple eye gaze paths and a small number of clear hot spots (centered on the desired conversion-related areas) are a good indication that the page will be more effective.”

Another way these can be used is in improving the effectiveness of print ads. My friends in the Community Law club at McGill Law are putting on an event dealing with depression in law students. Here’s their ad:

And here’s the heatmap:

Notice how little attention goes to the upper right call to action? When I first saw this posted on a bulletin board, I didn’t notice it at all. I thought it was strange someone would just post a random fact, so I looked around and only then did I notice the call to action.

A better ad would have probably made the call to action larger and placed it somewhere below the main copy.

Indeed, this bottom-left to forward-right layout, which I’ll call the forward slash layout ( / ), is very confusing to Western readers, because we’re accustomed to reading from left to right and top to bottom. As my fellow fans of Gord Hotchkiss and Enquiro B2b SEM and usability know, study after study has confirmed this pattern.

Now let’s see these heatmaps in practical web action. Let’s see what we can use them for!

Some sample heatmaps

Suppose you’re SpyFu. And you’re curious to find out your competitors’ conversion rates—for example, AdGooroo. Here’s AdGooroo’s landing page.

What jumps out at you? Personally, the toolbox in the middle grabbed and held my attention, which is bad news for the copy. Feng-GUI seems to agree:

So if you’re SpyFu, you might think that you’re laughing all the way to the bank. The problem is SpyFu’s landing page…

… isn’t so hot either.

The main call to action above the fold looks like some support link because of its placement near the login link, and the areas of the page that are emphasized are secondary (Live Help) and a fake button (Top Secret) that isn’t actually clickable!

For people considering media buys, and what creative to use, I have the [untested] belief that Feng-GUI can predict the best creative. How?

Have the graphic artists mock up a screenshot with your various banners in place. Then put each screenshot through Feng-GUI to see what draws the most eyes, and you might have found the ad that will draw the greatest CTR for you.

As an aside, I’d like to point out that the faces in the heatmap aren’t highlighted, yet eye-tracking reveals that faces tend to draw our eyes. Feng-GUI has an option to let it know there are faces in the image, and it does slightly better with this feature enabled. But nonetheless, the resulting heatmap isn’t great. So be aware that the tool isn’t 100% accurate; Feng-GUI itself only claims to accurately assess 75% of what actual eye-tracking would capture.

Ironically, Feng-GUI doesn’t seem to have used its predictive mapping on its own site. When I first tried buying credits for Feng-GUI, I went crazy trying to find the “buy now” button. Beside the fact that “buy now” is not in the main navigation, it’s not obvious on the products page either. Looks like I’m not alone in my frustration—I used the tool on the page and ended up with this result:

The order button is one of those small, unimposing white jumplinks at the top of the page. Exactly in that negligible row of links above the active window that gets no attention. And because Feng-GUI couldn’t be bothered to add a simple buy now button with Paypal, you need to email them to request to buy. Only then will they send you a Paypal money request.

By the way, there is a competing predictive heatmapping tool, Attention Wizard, currently in beta. I’m not spending time reviewing their tool because it currently only allows you to generate one heatmap a day.

Below is a comparison between the testing techniques mentioned, taking into consideration the overall use of the testing technique, coding needs, design needs, granularity of results and other considerations.

Most testing tools provide these options, but since Google Website Optimizer is a free tool that provides both options, it is a good place to start and try the examples I provide below.

A/B Test

An A/B test is the most common and easiest type of landing page test to conduct. It consists of creating alternative pages for a specific page and showing each of them to a certain percentage of visitors. For example, if you create 4 different variations of a landing page, 20% of visitors to the website will see each version (4 variations + original). Cookies are used to maintain a consistent user experience—if a visitor sees one version, they will see it again and again when visiting the website as long as the cookies are not deleted. Below is a representation of how this technique works.

Image created by Yam Designs. For a high res version of the image go to A/B test scheme.

To implement the test with Google Website Optimizer, scripts need to be included on the pages to be tested. These include:

• Two JavaScript codes on the original page: one that performs a redirect to the additional variations (head of the page) and one that measures the number of times the page was seen (this can be placed anywhere below the redirect code).
• One JavaScript code on each variation page to measure the number of visitors viewing each page.
• One JavaScript code on the conversion page to measure which visitors converted; this will measure the success of each page variation.

Advantages of A/B tests

Design freedom. A/B tests are often used to experiment page design options that vary dramatically, including position of text and pictures, background colors, number of pictures on the page, use of icons and navigation structure. Implementing such tests using the multivariate technique is possible, but it is technically challenging (but if you really want to do it, and you are technically savvy, see this post on the Google Website Optimizer Tricks blog.

Less JavaScript coding. as described above, the codes necessary to implement an A/B test are very simple and can be added to the website in a matter of minutes.

Faster results. A/B tests usually involve fewer combinations with more extreme changes; multivariate tests involve many more combinations and variations. In addition, since A/Bs show significantly different designs, the expected improvement of the page is usually higher, diminishing the time the test will run.

Multivariate test

Rather than testing different versions of web pages, as we do with A/B tests, Multivariate tests experiment with elements inside one specific page (for purists, we are referring to full factorial experiments, which is the method used by most testing tools). Basically, we define elements inside a page (e.g. a picture, a text or a button) and provide different alternatives of each element. The testing tool will show each element combined with all other elements to visitors. The resulting combinations are derived from the number of elements multiplied by the number of element variations. Just as with A/B testing, however, each visitor sees only one particular combination of elements regardless of how many times they view a page. Below is a representation of how this technique works.

Image created by Yam Designs. For a high res version of the image go to Multivariate test scheme.

In terms of coding, the programming a multivariate test is slightly more complex than a simpler A/B test. A few pieces of JavaScript code need to be implemented: one opening the test, one for each tested element and one closing the test. In addition, a JavaScript will be added to the conversion page to measure combination success.

Advantages of multivariate tests

Granularity of results. Since it is a full factorial experiment, multivariate tests show which elements are the best performing separately, as well as the correlation between the elements. This can be very useful when projecting the results to other parts of the website.

No redirects required. Since all elements tested are inside the page, there is no need to redirect from the original page to the tested pages. Although redirects can be performed smoothly, I believe it is better not to use them whenever possible, as they can slow the flow and affect user experience.

Fewer design resources required. Since we will be testing different designs with existing elements on a page, this will not require too much design effort.

Concluding, both types of testing have their own advantages and disadvantages. Each can be a perfect technique, depending on the needs of the website. They should always go hand-in-hand, using one to test completely different designs and the other to optimize the current design. The important thing is to understand that testing is not a one-time effort It is an ongoing exercise that should be part of the mindset of an organization. As Avinash Kaushik once wrote in his blog, Experiment or go home!

Websites are laboratories, not sculptures

This is the first principle when it comes to website optimization. People tend to think of their websites as a finished product, which was built taking into consideration customer needs and stakeholders requests. But isn’t the world changing? Isn’t it important to tap into new markets, new customers, new opportunities? Isn’t it always good to improve conversion rates?

The website should not exist solely to serve the needs and desires of the designer or the CEO of a company. It should serve the purpose of the customer; otherwise the CEO will be the only one visiting the website in the long run. In a recent two-part paper I wrote with Avinash Kaushik we propose a framework for Web Analytics 2.0: Empowering Customer Centricity (full pdf of Part II, to be published on Vol. 2 Issue 2 of SEMJ). We write about the benefits and best practices when it comes to testing. We believe that “the web analyst must try endlessly and learn to be wrong quickly; learn to test everything and understand that the customer should choose, not the designer or the website manager.”

How to choose a page to test?

With hundreds, thousands, or even millions of pages, how should you decide where to start? Which page, if tested, will bring the biggest increase in revenues? A good way to start is to perform a motion charts analysis on the Top Content report on Google Analytics. The following three metrics (which can be found in all web analytics tools) are highly useful when choosing pages to test:

Contribution to revenue. This metric can take different forms in different web analytics tools, but it basically tells you the contribution of each page to the overall revenue of the website. If the website has an ecommerce platform, this value should be some kind of purchase value divided by the pages seen before a purchase. If the website does not have an ecommerce platform, each goal should have a value linked to it so that we can calculate the contribution of each page to conversions.

Exit rate. This is the percentage of visitors that abandon the website from a specific page. A high exit rate shows that a page is not engaging enough and can be driving your customers away—unless the page is the “thank you for ordering” page, where it’s a good sign. If the objective is to test landing pages, the appropriate metric is the bounce rate, the percentage of visitors that leave the website without interacting with it. A high bounce rate shows that a landing page is performing poorly and should be optimized.

Average time on page. This metric can be an indication that visitors are having difficulty completing a task. For example, if you notice a very high time on page for one of the steps of a registration process, it might be a hint that users are having difficulties understanding what they are supposed to do—though this might not be true for content websites, where you do want your visitors to spend more time reading content.

These are the most common measures of success and failure of a page for most websites. It is important to consider all three metrics (and others that might be specific to your website) and prioritize pages to test. Once you know which pages are the most important, check which ones can be tested quickly. Go for the easiest! Once the first test is running and you tune the pages to improve results, management will implore for you to keep testing.

How long does testing take?

When choosing a page to test, it is also important to take into consideration what elements affect the time it will take in order to receive significant results. Basically, the variables that will affect the duration of your test are:

• Number of combinations tested: As the number of combinations increases, the duration of the test increases.
• Volume of traffic on the page: As the traffic of the tested page increases, the duration of the test decreases.
• Conversion rate of page: As the conversion rates of the page increases (as defined by the tester when planning the test), the duration of the test decreases.
• Expected improvement (the percentage by which you expect to improve the website): As the difference between the pages tested increases, the duration of the test decreases.

To estimate the duration of your test, you can use the Website Optimizer Duration Calculator, a very handy calculator provided by Google.

What should be tested?

As a web analyst, I feel the urge to say the magic words it depends! It depends on the type of website, it depends on the targeted public, it depends on your budget and it depends on that totally unscientific factor, the mood of your boss. Here are three things to test that can bring high benefits with low costs.

Calls to action. Too often calls to action are hidden, by a loaded page with too many graphic elements, by appearing below the fold or by a bad design choice (too small, faint color, or a button that does not look like a button). By improving the call to action and making it prominent on the page, you can sometimes boost your conversion rates drastically

Look and feel. Pages must be visually comfy, i.e., people should feel at home in your website. This way they won’t feel threatened and maybe they will become your friends, register for your newsletter and even buy something in the website. One way to do this is to test different images. Sometimes using a baby picture works; sometimes using a couple at the beach; sometimes even using something creepy, like an insect—depends what people are looking for on your website.

Copywriting. For visitors to seriously consider buying your product, signing up for your newsletter, whatever—you must have a good unique selling proposition. And people should see it as soon as view a page or they will leave. So make your unique selling proposition simple and prominent. Depending on the page you are testing, you should adapt the length of your text: for landing pages, short text usually works better; for product pages, go longer, telling visitors all the reasons why your product is the best in the market.

For a comprehensive list of testing elements and methods, you might take a look at the excellent book written by Brian Eisenberg and John Quarto-vonTivadar: Always Be Testing.

Testing tools

The online testing market is growing at a very fast pace, and there are quite a few players offering comprehensive solutions. Here are the biggest players—each has its own advantages and disadvantages. Check their websites to find the most suitable one for your needs.

• Google Website Optimizer
• Amadesa
• SiteSpect
• Autonomy Optimost
• Omniture Test & Target

Testing can bring huge benefits to any website. So start testing today, and don’t stop!

Beware butterflies and magic bullets

How much can you learn from landing pages?

Some landing page optimization experts will warn you about reading too much into the results of a particular landing page test. There are often multiple factors at play in a given experiment, and it can be difficult to precisely separate the different effects.

It’s a valid point, but taken to an extreme it becomes an argument for the “butterfly effect”—that a butterfly flapping its wings in Thailand might trigger an elaborate chain of events that dramatically alters the outcome of your experiments.

While that’s a fun philosophical debate, it’s not a practical position. As marketers exploring new ideas, we always have to deal with uncertainty—the secret of success is making educated guesses based on empirical—albeit imperfect—information.

On the opposite extreme, other folks claim that there are universal recipes that improve conversion rates in all situations—so-called “magic bullets.” These aren’t general best practices, such as “employ good visual design,” but rather specific formulas such as “use a green background,” “have an image of a smiling person”, and “include three one-line bullets.”

Warning bells should go off in any marketer’s head when they hear such one-size-fits-all recommendations without regard for the particulars of audience, market, or brand.

Rational landing page optimization

The middle ground between those extremes is what I dub the “rational” school of landing page optimization. There are three premises behind this approach:

• Different audiences, markets, brands, and campaigns have different characteristics
• All tests are not equal: different kinds of tests reveal different kinds of insights
• All confounding variables are not equal: some are more controllable, some have more influence.

The first premise dismisses the magic bullet approach. Selling a subscription to a pop music service is different than generating leads for a network storage solution. Even the network storage solution is—or should be—marketed differently to small-medium businesses (SMB) versus large enterprises. They have different desires, pain points, demographics/firmographics, etc.

As you dig deeper, you identify more and more segments in your market that respond to different marketing messages and presentations.

In rational landing page optimization, you embrace such segmentation in your marketing programs. After all, the big advantage of targeted search marketing with matched landing pages is that you can authentically engage different audience segments at the very top of your sales funnel.

This leads to three rules of thumb:

• Treat each segment as its own experimental space—look for learning within a segment.
• Don’t try to create “one page to rule them all”—pages are cheap; customers are valuable.
• Iteratively narrow your segments as long as doing so produces ROI—the digital world often rewards deep segmentation.

With such segmentation in mind, you can then consider four different kinds of tests—trivial, contextual, tactical and strategic—categorized by how much reusable learning they can provide:

Trivial tests. In this model, tests of different headlines or page colors are mostly trivial as far as reusable learning is concerned. That’s not to say that such elements might not have significant impact on a specific test—I’ve seen headline changes generate 50% lift—but rather that such factors are hard to reliably adapt from one set of circumstances to another.

Contextual tests. It’s at the next level up—with contextual elements—that you can start to form meaningful hypotheses. How much impact do seasonal themes have on your conversion rate? How much does the degree of specificity between the ad and the landing page impact the outcome?

To a certain extent, contextual tests are about discovering what would otherwise be confounding variables and systematically testing them. There’s still circumstantial sensitivity here, but useful and reusable patterns can emerge. For instance, is it worth tailoring your landing pages for seasonal factors?

Tactical tests. Higher yet are experiments to identify winning tactics within a segment. Tactical tests include things such as different offers, different levels of “depth” in the format of the landing experience (one page? a multi-step path? a microsite?), different data collection requirements in forms, etc.

These differences often have economic implications for both the respondent and the marketer—such as trading off the value of collecting additional information with the friction that a longer form imposes on the conversion process. In my experience, these type of tactics—when winning ones have been discovered—have a relatively high degree of portability from one landing page to another, at least within a segment for a particular company. You can learn what works here.

Strategic tests. At the highest level are strategic tests to identify new audience segments, the overarching value proposition for each segment, and the granularity of sub-segments within them. In multi-step landing pages, these tests may be conducted with different segmentation choices.

Often, however, strategic testing is about determining how many completely separate pages are optimal within a marketing program, each matched to a different slice of the audience. Honing in on new segments is possibly the greatest payoff from structured landing page testing, as those insights are useful not just in future landing pages but in other marketing vehicles as well.

If you disagree with what I’ve put in each category, feel free to adjust them to your own hypotheses—perhaps in your case color is a tactical choice? My overarching point is to construct tests with these different learning objectives in mind.

Measuring success

With a nod to the butterfly effect folks, it’s true that the reusable learning from these tests is hard to quantify precisely. However, rigorous dissection is not really your goal.

In rational landing page optimization, I would assert the following:

• Most tests are hypothesis-driven—you’re testing an idea from the start, not trying to fit explanations to the data after the fact. This is an important distinction.
• Especially with tactical and strategic tests, the number of simultaneous elements varied in any one test should be minimized, reducing interaction effects.
• Ideas that are believed to be applicable from one landing page to another will, by that belief, be tested repeatedly in a variety of circumstances; if they continue to correlate with high conversion rates over time, that belief is rationally reinforced.
• Ultimately, the proof is in the pudding—with rational landing page optimization, you expect to sustain improved conversion rates across many pages in many programs over time. If you’re successful by that metric, does it matter if the weights of contributing factors are somewhat fuzzy?

People in general, and marketers in particular, are very good at intuitive pattern recognition—in ways that are, frankly, hard to capture in oversimplified mathematical models. To be sure, this sometimes leads us astray, but more often than not it gives us a competitive edge.

Rational landing page optimization helps develop that intuition within relevant contexts and segments. What you learn won’t always be perfect, but it will give you momentum that can be measured in the net results.

Here’s a quick exercise, the Landing Page Wonder Wheel—as in, “I wonder how to improve my landing pages?”—that can give you fresh inspiration.

The Landing Page Wonder Wheel consists of eight dimensions on which you rate your current landing page creative and management capabilities, on a scale of 1 to 10. A 1 means you’re not doing very well there, while a 10 means you may be the best in the world at it.

1. Message match. How tight is the continuity between your adverting and your landing pages? If you run lots of ads across different keywords, but you’re driving everyone to the same few landing pages, then your message match probably isn’t very good. For example, if someone clicks on an ad for home refinancing, but they’re sent to a page that generically talks about mortgages, that’s not as tight as a page exclusively on refinancing.

Message match explicitly connects the dots for your respondents, instead of counting on them to hunt for and infer your relevance to their goal. Achieving good message match usually requires more specific landing pages and a process to keep them in sync as your advertising changes.

2. Visual design. How good do your landing pages look? From the high-level concept and layout, down to the details of execution such as fonts and image cropping, are your pages attractive? For most people who click on your plain text search ads, the landing page is where you will make your real first impression. Just as you probably shouldn’t show up to a job interview looking as if you rolled out of bed five minutes ago, tossed something on, and stumbled out into the world, you don’t want your landing pages to looks disheveled or uninspired either. This is a quintessential branding moment.

You may not be a graphic designer yourself—and if you aren’t, I wouldn’t recommend buying Photoshop and trying to fake it. Instead, find a great graphic designer who can make your pages beautiful. It doesn’t have to be a full-time position: a little quality design effort can go a long way, with page templates and a library of reusable image assets. Don’t downplay this though: in a competitive landscape, landing pages are in a beauty contest.

3. Depth. How much substance do your landing pages have? Depth is about delivering meaningful content rather than fluffy marketing-speak. Landing pages shouldn’t be superficial—otherwise they’re a waste of time. You want to share real information, tailored to the search that respondent was pursuing. Just pasting a dynamic keyword insertion (DKI) into your headline isn’t sufficient.

Depth doesn’t mean you should shovel a ton of content on to a single page though. Multi-step landing pages, where respondents drill down to the content and offers that are best aligned with their interests, can be highly effective. The key is to make sure that with each extra click, you live up to expectations, providing a deeper and more relevant experience. Microsites focused on a particular topic or idea can work well too. But ultimately depth is more about quality than quantity.

4. Freshness. How frequently do you revisit your existing landing pages to update them and inject new life? If you have stale pages out there from a year or more ago, then your freshness score is low. If, on the other hand, you systematically check your pages each month, your score should be climbing. This is more about landing page management than landing page design.

The basics of freshness are making sure that content and offers are current. There’s no surer way to damage your brand than to proudly present someone with an expired offer or a stale fulfillment piece (e.g., “fill out this form to receive our hot-off-the-presses 2006 research on the state of social media”). But freshness is also keeping your messaging up to date, recognizing that as your market evolves, your customers acquire new baseline knowledge, nomenclature, and shared cultural references. Even the look-and-feel of your pages can signal how on top of things you are, as the “fashion” of leading websites progresses from year-to-year. Bottom line: to keep respondents engaged with your landing pages, you need to stay engaged with them too.

5. Interactivity. Are your pages flat text and images, or do you provide interactive ways to capture a respondent’s attention? In the age of YouTube, a video can be a compelling way to build rapport. A Flash or AJAX widget that lets respondents click on tabs or thumbnails—or perhaps play with an animated diagram of your key benefits—can get them involved with a low hurdle. The secret is to incorporate these features as part of your design and messaging, not something garish or slapped on as an afterthought.

Social media is another way—albeit more experimental in this context—to add interactivity to your pages, such as bringing in Twitter feeds or Facebook Connect applications. You have to be careful about reinforcing your message and not distracting from it. But if you can use social devices to humanize yourself early with a new prospect, and coax them into a conversation, then you’re ahead of the curve.

6. Launch speed. How long does it take you, concept-to-completion, to launch a brand new landing page? Maybe there are technical or administrative hoops you have to jump through. Maybe you get held up waiting for someone to take the URL live, or add a tracking code to your checkout page. Maybe you just don’t have the time or resources. But whatever the reason, if you can’t deploy a new landing page as quickly as you can publish a new AdWords ad, then there’s room for improvement.

Landing pages should have the advantage of being quick, nimble, and inexpensive—a lightweight way to address niches across your market. As you build long tail (or even mid-tail) search campaigns, you want to follow through with message matched post-click marketing. But to achieve this, your per-page overhead needs to be low. If it’s not, track the time at each step along the lifecycle of your next landing page and start brainstorming: how could we speed this up?

7. Non-conversion value. How well do you do with the respondents who don’t convert on your landing pages? This may seem counterintuitive at first, but if your conversion rate is 20%—which would generally be quite good!—then what are you doing with the other 80%? After all, if they clicked on your ad, they demonstrated non-trivial intent. Just because they weren’t ready to convert on that specific offer at that exact moment, doesn’t make it a throwaway experience.

There are several ways to increase your value to non-converters. Maintain good brand standards—this is your chance to start building up neural pathways. Deliver useful content before the conversion point, telling people something meaningful that is relevant to their search. Always provide an “escape hatch”, even if it’s a subtle link at the top or bottom of the page, to let people jump to your primary web site. (These are good principles for conversions too.) Have them leave remembering you in a good way.

You can also derive value from non-converters by analyzing what they do. For instance, in the context of multi-step landing pages, you can track which choices people click on as a simple type of behavioral segmentation. Learning which segments aren’t converting gives you the insight to make targeted improvements.

8. Boldness. Do your landing pages charge forward with bold, new ideas—or are they tepid and formulaic? Landing pages can be a fantastic sandbox in which to experiment with gutsy offers, spirited language, and vivid presentations. Since any given landing page handles only a sliver of your traffic—and because it’s usually easy to do A/B testing in this context—you can push the envelope without taking big risks. If a daring idea doesn’t pan out, you can quickly pull it down. If it catches fire (in a good way!), then you can expand its reach.

The case for boldness—aside from the timeless proverb that fortune favors the bold—is two-fold. First, in a competitive situation, where respondents are also clicking on rival ads, you want to stand out from the crowd. Not in a freakish way, mind you, but in a confident and creative way. Second, as you move further down the long tail, you end up outside your company’s well-worn messaging. The only way to discover what resonates with new market segments is to try some new ideas. Don’t be afraid to be creative—be more afraid of being dull.

How good is your wheel?

Now that you have your self-assessment scores, mark them on the wheel on each corresponding spoke, moving outwards for higher scores. So a 1 would be placed near the center of the wheel, while a 10 would be placed on the outer rim.

Next, connect the dots. What does it look like?

If your connected wheel doesn’t look very round, or if it’s rather small, then you should at least have a clear idea of what you can do to improve your landing pages. If you’re committed to tackling those challenges, then you can redo the wheel in 30, 60, or 90 days to see your progress—and correlate it with your conversion rates.

Might not hurt to do this exercise on some of your competitors either.

I hope this helps you launch some wonderful post-click marketing.