Custom variables are useful for all kinds of segmentation tasks as well as for adding information to your Google Analytics profile not normally captured by the tracking code. If you have customized your analytics install, there is a fair chance that you are using _setVar. It would be tempting, from reading the announcement to think that _setCustomVar is simply an improvement to _setVar that allows you to set the scope and manage more variables.

It is important, however, to realize that there is a crucial limitation to _setCustomVar that was not there for _setVar:

_setCustomVar(index, name, value, opt_scope)

From Google’s documentation on setCustomVar:

“There is a 64-byte character limit for the name and value combined…”

This throws up a number of a challenges for migrating from _setVar to _setCustomVar. There is the obvious one of not being able to store as much data which is annoying, but subject to persuading the Google team to reverse a design decision there’s nothing you can do about it (what’s the problem guys? Not enough server space to store a few more bytes in that string?). What’s more tiresome is that it’s non-trivial to tell how long your string is because Google URL encodes the string on the way into the system, changing its length! (Note that this is not in the documentation…).

Don’t panic, though! I have a work-around that will hopefully save you from some of the heartache I have been experiencing over the last few days.

The best way I have found to solve the problem is to:

• Give your variable names that are as short as possible—preferably only one character (comment the code to remember what you meant by variable “p” later!)
• Take the string you want to insert into a custom variable and encode it (using the JavaScript function encodeURIComponent)
• Truncate the encoded string to 64 characters minus the length of your variable name
• Decode the string (using the JavaScript function decodeURIComponent)
• Pass this string to Google Analytics, where it will be URL encoded again on the way in.

Or, you could simply use my function distilledTruncate(myString) assuming you are using 1-character variable names after including this JavaScript file above your analytics customization code:

<script type="text/javascript" src="http://attributiontrackingga.googlecode.com/svn/trunk/distilled.FirstTouch.js"></script>

Note that this JavaScript file also gives you a couple of other functions to help set up first touch tracking in Google Analytics—something I have written about more on our own site. I am undoubtedly going to improve this code, but if you would like to avoid the possibility of me including something malicious in there, you can link to a static version:

<script type="text/javascript" src="http://attributiontrackingga.googlecode.com/svn-history/r3/trunk/distilled.FirstTouch.js"></script>

It is important that you perform this kind of truncation to avoid passing a string that ends up longer than the limit as _setCustomVar itself will not truncate the input for you—it will simply fail to record a value and return false. It is very likely that this will simply result in missing data in your analytics profile. This kind of relatively silent failure is quite dangerous on an upgrade path like this in my opinion.

If you have any suggestions for improvements to my JavaScript, I have released it under a GPL license and you can report issues here. Feel free to let me know any “gotchas” you have found in the comments!

The example I want to discuss today is segmenting your tests to see how they perform for different kinds of users of your site. I’m going to show you a few ways of gathering GWO data in Google Analytics (GA) to bring the full power of the GA platform to your GWO testing.

A classic example is testing the impact of changes to the call-to-action for new vs. returning users. These two segments of users often have very different requirements and we can see why they might react differently to different enticements. For someone who knows your brand and is familiar with your offer, a discount could be supremely effective while a new visitor might respond better to long copy explaining the benefits (this is pure speculation and is designed only to serve as an example). For sites that get high numbers of returning visits (e.g. eBay), returning visitors are a self-selected set of people who understand how to use the control and/or have struggled through the learning curve. They may be resistant to change and prefer the control despite there being room for improvement for new visitors who haven’t gone through that learning curve.

Other useful segmentations during conversion rate tests include:

• Traffic source (e.g. organic search vs. paid search vs. referral traffic)
• Country/location
• Whether a user has visited a specific page on the site or in the conversion funnel
• And a whole load of other things

There are three basic steps to getting the data you need:

1. Ensure that you are able to track GWO variables in GA
2. Set up appropriate filters and avoid contaminating your existing profiles and data
3. Analyse the data

#1 Track GWO Variables

Google Analytics has a (relatively little known) function called utmx to return basic information about a GWO test to analytics. By tracking pageviews that include information in the URL about the test and variant being run, you can set up a profile within GA that has an understanding of conversion rate tests. There is more information on how to do that in this quite technical guide. It may also be useful to read this alternative approach from ROI Revolution that uses a custom ga.js file.

Tracking a pageview such as /my/page.html?gwo_exp=1234567890&gwo_var=0 gives all the information needed to create a profile that stores the variant of the test in a custom variable.

#2 Avoid contaminating your main profile

I suggest implementing the tracking of GWO variables in a new profile. In your core profile, you will probably want to strip out this information so that you can see all visitors of a single page concept together (i.e. not splitting out by which variant they saw).

In this example, you would want a filter such as (from the guide referenced above):

• In “Field A -> Extract A”, select Request URI and (^.*)(.gwo_exp.*)
• In “Output To -> Constructor”, select Request URI and $A1

#3 Analyse the output

One of the nicest things about GWO is its ability to abstract away the maths behind determining winners. While that functionality is obviously still available for the aggregate test (on the whole set of users) there is an additional step required to work out whether you are seeing statistically-significant results on segments of your users.

Although learning the statistics behind these calculations is fun(warning: your experience may differ), we are primarily looking to get to the answer here. For this I recommend using an online sample size calculator. In order to analyse, for example, whether your test has reached statistical significance on new users only, you would segment the data to show only new users and find the number of users in this segment who had seen each variant and the number of each of those who had converted (for whatever conversion goal you are seeking).

With this data in hand, you can fill in the fields in your chosen calculator and, see the relative conversion rates. For most tests, you would seek a 95% confidence that a variant was better than the control. Consider the following data:

These results suggest that you would be happy declaring variant B better than the control (confidence > 97%) but even though variant A has a higher conversion rate than the control at present, you would not be confident declaring it better yet.

What I’d like to have shown is real-world examples of the ways that tests can show misleading things when you look at the average data when digging into the segmentation reveals insights about different classes of user. Unfortunately, I don’t have any public examples. If anyone has real-world examples they can share in the comments, I’m sure the SEL community would love to discuss them.