Note: By submitting this form, you agree to Third Door Media's terms. We respect your privacy.

A Primer On Website Testing

Daniel Waisberg on September 19, 2009 at 5:08 pm

Search marketers can learn a lot from scientists. Scientists spend their life testing things, one after the other, incessantly trying to discover new interactions between atoms, molecules, viruses, bacteria, etc. One of the greatest scientists of all time, Albert Einstein, said, “A theory is something nobody believes, except the person who made it. An experiment is something everybody believes, except the person who made it.” Want everyone to believe in your website? “Experiment” with it—in other words, test it and tune it for optimal performance.

Websites are laboratories, not sculptures

This is the first principle when it comes to website optimization. People tend to think of their websites as a finished product, which was built taking into consideration customer needs and stakeholders requests. But isn’t the world changing? Isn’t it important to tap into new markets, new customers, new opportunities? Isn’t it always good to improve conversion rates?

The website should not exist solely to serve the needs and desires of the designer or the CEO of a company. It should serve the purpose of the customer; otherwise the CEO will be the only one visiting the website in the long run. In a recent two-part paper I wrote with Avinash Kaushik we propose a framework for Web Analytics 2.0: Empowering Customer Centricity (full pdf of Part II, to be published on Vol. 2 Issue 2 of SEMJ). We write about the benefits and best practices when it comes to testing. We believe that “the web analyst must try endlessly and learn to be wrong quickly; learn to test everything and understand that the customer should choose, not the designer or the website manager.”

How to choose a page to test?

With hundreds, thousands, or even millions of pages, how should you decide where to start? Which page, if tested, will bring the biggest increase in revenues? A good way to start is to perform a motion charts analysis on the Top Content report on Google Analytics. The following three metrics (which can be found in all web analytics tools) are highly useful when choosing pages to test:

Contribution to revenue. This metric can take different forms in different web analytics tools, but it basically tells you the contribution of each page to the overall revenue of the website. If the website has an ecommerce platform, this value should be some kind of purchase value divided by the pages seen before a purchase. If the website does not have an ecommerce platform, each goal should have a value linked to it so that we can calculate the contribution of each page to conversions.

Exit rate. This is the percentage of visitors that abandon the website from a specific page. A high exit rate shows that a page is not engaging enough and can be driving your customers away—unless the page is the “thank you for ordering” page, where it’s a good sign. If the objective is to test landing pages, the appropriate metric is the bounce rate, the percentage of visitors that leave the website without interacting with it. A high bounce rate shows that a landing page is performing poorly and should be optimized.

Average time on page. This metric can be an indication that visitors are having difficulty completing a task. For example, if you notice a very high time on page for one of the steps of a registration process, it might be a hint that users are having difficulties understanding what they are supposed to do—though this might not be true for content websites, where you do want your visitors to spend more time reading content.

These are the most common measures of success and failure of a page for most websites. It is important to consider all three metrics (and others that might be specific to your website) and prioritize pages to test. Once you know which pages are the most important, check which ones can be tested quickly. Go for the easiest! Once the first test is running and you tune the pages to improve results, management will implore for you to keep testing.

How long does testing take?

When choosing a page to test, it is also important to take into consideration what elements affect the time it will take in order to receive significant results. Basically, the variables that will affect the duration of your test are:

Number of combinations tested: As the number of combinations increases, the duration of the test increases.
Volume of traffic on the page: As the traffic of the tested page increases, the duration of the test decreases.
Conversion rate of page: As the conversion rates of the page increases (as defined by the tester when planning the test), the duration of the test decreases.
Expected improvement (the percentage by which you expect to improve the website): As the difference between the pages tested increases, the duration of the test decreases.

To estimate the duration of your test, you can use the Website Optimizer Duration Calculator, a very handy calculator provided by Google.

What should be tested?

As a web analyst, I feel the urge to say the magic words it depends! It depends on the type of website, it depends on the targeted public, it depends on your budget and it depends on that totally unscientific factor, the mood of your boss. Here are three things to test that can bring high benefits with low costs.

Calls to action. Too often calls to action are hidden, by a loaded page with too many graphic elements, by appearing below the fold or by a bad design choice (too small, faint color, or a button that does not look like a button). By improving the call to action and making it prominent on the page, you can sometimes boost your conversion rates drastically

Look and feel. Pages must be visually comfy, i.e., people should feel at home in your website. This way they won’t feel threatened and maybe they will become your friends, register for your newsletter and even buy something in the website. One way to do this is to test different images. Sometimes using a baby picture works; sometimes using a couple at the beach; sometimes even using something creepy, like an insect—depends what people are looking for on your website.

Copywriting. For visitors to seriously consider buying your product, signing up for your newsletter, whatever—you must have a good unique selling proposition. And people should see it as soon as view a page or they will leave. So make your unique selling proposition simple and prominent. Depending on the page you are testing, you should adapt the length of your text: for landing pages, short text usually works better; for product pages, go longer, telling visitors all the reasons why your product is the best in the market.

For a comprehensive list of testing elements and methods, you might take a look at the excellent book written by Brian Eisenberg and John Quarto-vonTivadar: Always Be Testing.

Testing tools

The online testing market is growing at a very fast pace, and there are quite a few players offering comprehensive solutions. Here are the biggest players—each has its own advantages and disadvantages. Check their websites to find the most suitable one for your needs.

Testing can bring huge benefits to any website. So start testing today, and don’t stop!

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.

About The Author

Daniel Waisberg

Daniel Waisberg is the Principal og Conversion Journeyand the founder of Online Behavior, a Marketing Measurement & Optimization website. He holds a M.Sc. in Operations Research and Decisions from Tel Aviv University, where he developed a statistical model that helps to optimize websites using Markov Chains. Daniel is a frequent speaker & member of the Advisory Council of the eMetrics Marketing Optimization Summit. You can follow him on Twitter or Google+.