“Let’s say this is your first day at Urbanspoon and I show you the following data. We’ve just launched an A/B test of that I’d like you to evaluate. [The example can be almost anything you want to test different results for – from almost any search element, PPC campaigns, email subject lines etc. In this case, I’m using a PPC example.] Imagine you are running two different ads on a campaign with 50 kewords. We’ve been running Ad A for a while and have 17,235 impressions and 272 clicks. I started running Ad B last week and that has received 41 clicks on 2,253 impressions. What would you do?”

I’m looking for an answer that goes beyond demonstration of pre-algebra skills and rudimentary familiarity with a calculator.

Obvious answers include splitting up 50 keywords into different groups, looking down stream to see differences in conversion rates, and technical answers around quality score. But what I’m really looking for is a theoretical understanding of statistics and the interplay between sample sizes, variability and confidence intervals.

Answers to the above theoretical question usually fall into one of three buckets:

1. I’d run as more Ad B’s so our impressions are equal and then compare the click through rates.  #FAIL
2. I’d run the Ads longer, you need at least 3 weeks of data to make a decision.  #FAIL
3. Ad B is better b/c the click through rate is higher.  #FAIL “and thanks for taking the time to talk with me, our HR department will be in touch . . .”

Turns out, you don’t need to have an equal number of impressions or a set amount of time to run this analysis. It’s actually a fairly simple concept that can ultimately then be mathematically defined:

The greater the difference between your A and B samples (drawn randomly from the same pool) the smaller the size your test needs to be in order to confidently assert that one performs better than another. Example – if we wanted to test if men were taller than women and we measured 100 men and 100 women  and the men averaged 7 feet tall and the women averaged 4 feet tall, you’d be fairly confident saying that men are taller than women.

Conversely, if the difference was 3 inches instead of 3 feet, you’d probably want to measure more men and women before confidently asserting men are taller than women.

In fact, it’s possible that your sample was misleading – as a population women are really taller than men, but your sample didn’t bear that out. This level of confidence can be mathematically expressed as a percentage – I’m 95% certain that A is better than B. (Meaning there is a 5% chance, or 1 out of every 20 times where you’ll unwittingly pick the underperformer.) The greater the level of confidence you want, the larger the sample size you need.

All of this can be calculated with innumerable free online tools. Larger sophisticated systems like Adwords and big ESPs build this statistical testing in to their testing methodology – but it’s easy for do-it-yourselfers too.

I like a tool called AB Tester, which allows you to measure up to three alternatives compared to a benchmark:

In the results above, I’ve done the analysis for our question . . . The “Confidence” column tells me there’s a 79.19% chance that B is better than our control A.

Watch how this Confidence grows when we add a zero to each column – keeping the CTR the same but increasing the sample size:

By increasing the size of the test tenfold, now there’s only 0.5% that A is really better than B.

Let’s go from theoretical to real. Here are results from an email test we did for our Hawaiian getaway promotion to Ludobites 9. (It’s over now, sorry.)

The first data column is sends, then delivered, then opens, then clicks. Assume we want to test three different content types to three different cities (now admittedly this is not a random sample – maybe people in San Francisco respond differently to content . . . )

Take the data from the 3/6 send and plug it in to A/B Tester. Note I’m comparing the CTR from Opened emails to isolate content as an impact to click through rate. Also note that while the sample sizes are similar, they don’t have to be the same.

My best performer here is the San Francisco content at a 5.5% CTR. I use that as a control and plug the other two into AB Tester:

This tells me there’s a 3.3% likelihood that the LA content might really outperform the winner (San Francisco). Additionally, there’s a 23.5% chance Seattle content is better than our “winner”. More testing necessary . . . .

It is possible to properly attribute blekko search traffic referrals by implementing advanced Web analytics configuration techniques.

Is the extra effort worth the trouble? Certainly investors are betting on blekko. In late September, blekko received an addition $30 million investment, half of which was from Russian search engine Yandex.

Why Web Analytics Systems Can’t See Blekko’s Search Keywords

The underlying difficulty many Web analytics systems encounter in recognizing blekko as a search engine is due to a reliance on search engines to specify the search keywords in the request for the destination webpage by using parameters in the referrer URL.

Unfortunately, blekko deviates from what has historically been the de facto standard for transmission of search query information. In the absence of advanced tool configuration, traffic from blekko will show up in a referring sites report instead of an organic search traffic report.

While this scenario is better than nothing, it makes keyword analysis difficult. Knowledge and analysis of search query keywords is a key element of search marketing as a user’s search keywords express their intent when selecting a site from a listing of search results.

In the following discussion, Google Analytics will be used to illustrate solutions, but the same data transformation logic can be applied to other tools as well. The technically inclined are encouraged to read on. Everyone else might want to pass this article on to their technical staff for analysis and implementation.

Deconstructing & “Fixing” The Blekko Search Referrer

A typical blekko referrer URL contains blekko’s domain name, the parameter ws and the user query. Spaces between multiple query words are replaced with plus signs.

In the following example, the user’s search keywords stop+blekko+indexing follow http://blekko.com/ws/ in the search engine results page URL, visible in the browser address bar.

Figure 1: A blekko search result with user keywords in the result URL

Unfortunately, this isn’t in the form most Web analytics systems were designed to recognize when processing organic search traffic.

The URL piece ws/ needs to be expressed as a parameter, in the format name = value, i.e. ?ws=stop+blekko+indexing.

Thus: http://blekko.com/ws/stop+blekko+indexing needs to be transformed into http://blekko.com/?ws=stop+blekko+indexing

Two Possible Solutions To Solving The blekko Attribution Problem

There are two potential solutions to proper recognition of blekko as a search engine by Web analytics systems. The first approach is to modify the raw data sent to the Web analytics data collection system. The second is to transform the data collected by the Web analytics system before it is processed for reporting.

Option 1: Modify The Search Referrer Sent To Google Analytics

The blekko referrer URL needs to be transformed into the de facto standard format which includes the user query using URL parameters, e.g. ?query=keywords, a format recognized by most Web analytics systems. JavaScript is well suited to this type of task.

The HTML standard doesn’t allow direct manipulation of the document (page) referrer, but the referrer sent to the Google Analytics data collection system can be changed by assigning the modified value to a special parameter, _setReferrerOverride.

In the asynchronous Google Analytics tracking code which follows, the code in bold will transform blekko referrer URLs and assign the result to _setReferrerOverride.

The code:

• assigns the page referrer to the variable blekref
• tests to see if blekref contains the string http://blekko.com/ws/
  • if so, tests to see if http://blekko.com/ws/already contains URL parameters, which start with a ?
    • if so, substitutes the ? with a & which indicates successive URL parameters
  • substitutes ws/ with ?ws=
  • places the modified URL in the Google Analytics variable _setReferrerOverride

<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount','UA-xxxxx-1']); var blekref = document.referrer ; if (~blekref.indexOf('http://blekko.com/ws/')) { if (~blekref.indexOf('?')) { blekref = document.referrer.replace('?','&'); } blekref = blekref.replace('/ws/', '/?ws='); _gaq.push(['_setReferrerOverride',blekref]); } _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Note: this code will need to be modified if blekko changes their URLs, such as using www.blekko.com in addition to blekko.com.

As with any code changes, do be sure to consult current Google documentation and fully test changes before putting them in production. JavaScript errors in a page can stop successive JavaScript from being executed. No warranties are provided, implied or otherwise!

Tell Google Analytics How To Recognize blekko As A Search Engine

Once blekko’s referrer string has been transformed into the standard query parameter = keywords format, the second and final step is to add blekko to Google Analytics’ list of known search engines.

This also has to be done within the Google Analytics tracking code inserted in each page of the website.

The syntax to add a search engine is of the form:

_gaq.push(['_addOrganic','blekko','ws']);

where blekko is a string in the referrer domain and ws is the query parameter name.

This has to be inserted somewhere before the ['_trackPageview'] call, i.e. from the above example we can change

_gaq.push(['_trackPageview']);

to

_gaq.push(['_addOrganic','blekko','ws'],['_trackPageview']);

A more in depth discussion on working with the _addOrganic configuration option can be found in an article on Google Analytics search engines list customization.

Tracking Page Speed In Google Analytics?

During testing, it emerged that Google Analytics would ignore the value of _setReferrerOverride if _trackPageLoadTime was specified.

Using the Google Analytics debugger, it became clear that Google was deprecating _trackPageLoadTime so the message is simple: remove _trackPageLoadTime from existing tracking code to avoid unforeseen consequences.

Option 2: Transforming Collected Data

Should sites not be able to modify their Google Analytics tracking code, or should their Web analytics system not provide a referral override option, there is a second approach which should work as well. The technique involves transforming the collected data before reports are created.

In Google Analytics, Profile filters perform this task. Filters are a wonderful, but underutilized, Google Analytics feature and this is a great example to show off their potential. The process is:

1. Identify search referrals from blekko
2. Update the medium, source and term campaign fields accordingly.

Unfortunately, Google Analytics only allows a profile filter to update one field at a time, so it will be necessary to create multiple filters.

To do this, enter the Google Analytics administration area and create new advanced custom profile filters as illustrated below. Filters can be created at the account level, making it easy to quickly apply the filters to multiple profiles.

Figure 2: Blekko set campaign medium filter.

The above filter selects referral traffic from Blekko which contains /ws/ in the URL and changes the Google Analytics Campaign Medium from referral to organic (required for blekko traffic to appear in the organic search report).

Currently, it isn’t strictly necessary to include the Field B requirement, but it may be helpful in order differentiate from paid search should Blekko offer it in the future. Note that the values referral and organic must be in English, even if the Google Analytics interface is in a different language.

Figure 3: Blekko set campaign source filter

Google Analytics will automatically identify blekko.com as the source of traffic, however, some may want to see the source report as “blekko” instead of “blekko.com” in their reports, so as to be consistent with the other search traffic sources. The above filter, which is optional, will do just that.

Figure 4: Blekko set campaign terms filter

The set campaign terms filter captures the user’s search query and places it in the Campaign Term field, the source of the keywords which appear in the Google Analytics organic search traffic report.

The key is in the regular expression specified in Field A:

^http://blekko\.com/ws/(.+)\??

It says, select referrers which start with http://blekko.com/ws/, capture everything that follows up to a question mark, which may or may not be present, and put the result in the campaign terms field.

Astute readers might note there is still one problem remaining: multiple keywords will still be separated by one or more plus signs which should be converted to spaces.

Unfortunately, it doesn’t seem that Google Analytics supports global (multiple) substitution using filters, but in the spirit of crowd sourcing, a reader may want to chime in to improve this last step.

One work around might be to create multiple substitution filters which will replace a single + with a single space:

Figure 5: Filter to substitute one plus sign, +, in blekko search keywords, with a space

Once the profile filters are created, they need to be activated by apply them to each Google Analytics profile.

The order in which the filters are applied is not important, with the exception of the plus sign substitution filters, which must come after the set campaign term filter. A Google Analytics best practice is to keep at least one profile without any filters as once data has been transformed by filters as there isn’t any way to revert filter changes.

An unadulterated profile is also helpful in troubleshooting any issues which may arise. New filters can (and should) be tested on a new “throw-away” profile before being applied to production profiles.

A Note About blekko’s Slash Tags

One of Blekko’s distinguishing features is the ability, encouraged by blekko, of users to further restrict their search queries using search operators and filters, what blekko calls slashtags. As the slash tags do condition the user’s query, and to a degree express the user’s intent, neither of the approaches illustrated above attempts to remove slashtags, if present, from the search query keywords.

Psychology has shown that many of our day-to-day behaviors are deeply related to our personalities, beliefs, and cultures. In the same spirit, I believe we can learn about people’s professional inclinations from their analytical behavior.

Google Analytics & A Tour Of Jerusalem

Recently, I presented at a Google Event in Israel on the subject of Analytics and Testing. Following the event, I traveled to Jerusalem with Timo Josten, Google Analytics Partner Program Manager for EMEA. As we approached Jerusalem, I decided to build the tour based on his Google Analytics preferences.

I asked him what is the first report he looks at when checking a new Google Analytics account, his answer: the Traffic Sources report. That’s because he believes that the sources of traffic tell a lot about the company’s marketing efforts and about the visitors themselves.

Based on that, we started by visiting the old city alleys (image below), as I understood that he is the kind of person that dives into the numbers to understand the visitors and then zooms out to understand the big picture. We started from inside and then moved out to a bird’s eye view.

In a different instance, I also had the pleasure of taking Bryan Eisenberg to Jerusalem. I asked a similar question: “what is the first report you look at when looking at a new Web Analytics account?”

Bryan explained that he usually looks at the big picture and then zoom in to see the details and understand visitor behavior better. Based on that, I took him to a spot where we could view the whole city and then we “zoomed in” to the old city alleys.

In summary, I believe that good analysts can be divided in two types of people: those who start broad and then drill down with the big picture in mind, and those who start on the details and broaden their view with a good notion about the numbers.

Google Analytics Psychological Test

People usually believe that one person’s friends can tell you a lot about that person’s beliefs and characteristics. Expanding on this same idea, I believe that the Web Analytics industry should promote the saying: “tell me what is your preferred report, and I will tell you who you are.”

In a previous post, I discussed the Web Analytics Process and the importance of building Key Performance Indicators that are customized based on a person’s interest. Below I provide a few personalities that can be discovered from Web Analytics behavior analysis.

The Hawk – Dashboard View

According to Wikipedia, “Hawks are widely reputed to have visual acuity several times that of a normal human being.” This enables them to fly very high and still be able to find the preys.

The same is true for analysts that focus on the so called bird’s eye view: they analyze the big picture and are able to find insights that will show where to focus the attention.

This view is especially important for online marketing managers or website owners, people that want to know the bottom line but also additional success metrics (such as engagement, campaign success, and visitors return rates).

The Slot Machine – Ecommerce View

When a person goes to a casino, he or she has one main objective: to make money. Especially slot machine addicts. This is not to say that they do not have fun along the way, but they are result driven and they will hardly move their focus.

Some people, when looking at Google Analytics, cannot move their eyes from the ecommerce reports. This view is especially important for senior managers or sales people, who mainly want to check the website results and compare it to other channels.

The Psychologist – Content View

Some people like to understand what other people think and like. Those people usually become psychologists, but once in a while they miss the target and become web analysts. Their main interest is to understand what visitors are reading on the website and if they are enjoying it.

The content reports are especially important for content specialists, editors, and analysts looking to understand what is the most engaging material on the website (which will bring insights into how to organize the website and where to advertise).

The Historian – Sources View

Historians are known for researching the past and trying to understand how we got where we are, and if history is likely to repeat itself. Traffic Sources tells us just that: how people got to the website (sometimes even their intentions via keywords) and how visitors from each background behave.

Traffic sources are especially important for marketing managers looking to understand how marketing campaigns are working and for SEM managers looking to optimize SEM campaigns.

The Geek – Technology View

Geeks just can’t help themselves: they will open Google Analytics and go straight to the Operating Systems and Browsers report. With Google Analytics v5 it is even easier, as there is a whole set of reports named Technology.

As the name says, these reports are very important for IT personal in order to understand which technology visitors are using and how to optimize website performance for that. In addition, it might also be important to senior management to drive decisions regarding mobile strategy (see Mobile Marketing for E-commerce).

Closing Thoughts

In summary, I believe that online professionals have different ways and interests when looking at data. It is possible to understand more about a person’s perspective by asking questions related to how he or she approaches and analyzes the data.

There are plenty of social analytics tools, but the most full-featured ones are often too expensive (i.e., geared towards enterprise-level users) and the inexpensive ones usually offer limited functionality.

And then there’s Topsy Social Analytics, which was just launched this morning and immediately becomes one of the best free Twitter analytics tools I’ve seen. There are a number of ways to use it and, if you click that link to check it out, don’t be surprised if you spend the next hour there.

How Topsy Social Analytics Works

With Topsy Social Analytics, you can analyze domains, Twitter usernames, or keywords — and they can be compared over four timeframes: one day, a week, two weeks or a month. Each type of analysis returns different results. Compare three domains, for example, and Topsy analytics will chart the number of links it found to each. Type in three keywords (including names, as in the image at the beginning of this article) and Topsy shows how many times each was mentioned. And if you compare up to three Twitter usernames, Topsy shows how many replies/mentions each received.

Have a look at the three different types of comparisons:

Topsy says it’s only counting “mentions that matter,” which they define as tweets that have a link or have been retweeted and don’t come from “bots or spam sources.” There’s no additional info on that, at least not that I’ve been able to find so far.

Scroll down the page after any search comparison, and Topsy also shows the top links in the past 24 hours that are related to your comparison — along with several data points that measure the performance of each link, such as influence, “momentum,” “velocity” and “peak.”

Marketers will like this for competitive analysis: Put your domain and two competitors’ domains in the comparison tool and see which links on each site have been most popular. And, it’s a bit hard to see in the screenshot above, but there’s also a link to download the link data in CSV format.

All of this is free. Topsy says an “expanded analytics offering” is in the works for later this year; I’d expect that to be a paid tool, but Topsy’s announcement doesn’t say one way or another.

Topsy has, for some time now, claimed to have the biggest index of tweets dating back to May 2008 — an index that was recently reported to be around eight billion tweets.

With OneRiot and others getting away from real-time/Twitter search, this kind of analytics tool falls increasingly on either Topsy or Twitter itself to provide. Topsy has put its first stake in the ground. Twitter also has its own analytics tool, but that’s currently only available to Twitter advertisers. There have been rumors that Twitter would also offer analytics to all users, but that hasn’t materialized yet.

Search marketing platform Kenshoo earlier this summer introduced what it calls Call Conversion Optimization. The program, according to Kenshoo, automatically adjusts bidding in response to ads that are driving calls rather than resulting in just clicks. Unique phone numbers are used, at a campaign or keyword level, to determine which ads are working from a calls standpoint.

Today Kenshoo further refined the program with partner Mongoose Metrics. Now unique phone numbers can be generated for each keyword — and individual user — so that no two people searching on the same keyword will see the same phone number on the resulting landing page. According to the press release:

The tight API integration of Mongoose Metrics’ call tracking and speech recognition technology combined with KENSHOO Local’s sophisticated bid optimization algorithms, known as Call Conversion OptimizationTM (CCO), allows advertisers to leverage Dynamic Number Replacement (DNR) to display a unique tracking telephone number to each user, creating a link between campaigns, keywords, and the converting call. Unlike other solutions, KENSHOO’s CCO tracks these conversions and suggests optimal bids for the keywords that generate phone conversions and suggests reduced bids or removal of keywords that generate clicks – and cost – but do not result in the desired conversions.

Kenshoo says Mongoose is going to add call recording for additional capabilities (e.g., keyword research) in the near future.

Yext, which describes itself as a “pay per action” platform (using calls) uses speech-to-text data mining to filter good phone leads from weak ones and says it only charges clients for good calls accordingly. This is an advancement over call-based advertising that relies solely on call duration for advertiser billing.

Marchex, which operates a pay per call exchange, has bet its business increasingly on phone-based advertising and call analytics. The company last week introduced “call mining,” which takes recorded calls and speech-to-text transcription and makes all sorts of interesting analytics options available on top of that via a dashboard.

According to the company’s promotional materials the following are the kinds of things that can be done or determined:

• On-the-phone conversion rates and caller intent
• Identify the channels, ads and keywords that drive the highest on-the-call conversion and return on investment
• Map call outcomes to parameters such as geographic location and gender
• Understand the effectiveness of call-based marketing campaigns
• Audit call center efficiency and caller experience
• Directly ascertain customer needs and pain points
• Flag calls by topic of discussion

Beyond identifying which keywords generated more calls, these capabilities reveal the quality of the specific calls and enormous amounts of additional information that can be factored back into marketing campaigns (as well as operations more broadly). Marchex EVP Matthew Berk and I discussed scenarios where national advertisers might adjust campaigns (ad copy, keywords) on a regional basis according to data revealed by the content of successful phone conversions.

At a higher level these tools point the way toward the convergence of marketing and customer service, which have historically been distinct silos within organizations.

Call measurement firm Telmetrics told me yesterday that the company has seen “exponential growth” in demand from online marketers as they realize the analytics and data-mining benefits of tracking and monitoring calls.

I’d like to hear whether people believe the SEO concerns remain; however Telmetrics said it has figured out a way “through automated integration and data reconciling” to prevent tracking numbers from appearing as primary local business lines.

The larger point is that these optimization and call mining capabilities offer marketers new levels of insight into the effectiveness of campaigns that were just not possible with clicks and conventional web analytics in the past. I also believe — and I’m totally speculating here — that Google will add call analytics to AdWords (either on its own or through an acquisition) in the next six months. We’ll see.

What’s The Referrer?

When you visit a web page, by default, every major browser (Internet Explorer, Firefox, Chrome…) reports the last page that you viewed before clicking over to the current page you’re viewing. IE, what page “referred” you to the current page. This information is known as the “referrer URL.” (Technically it’s the “referer” string, because of a misspelling in the HTML technical specs years ago).

For example, let’s say that you did a search on Google for “google pac-man,” the game that Google debuted last week. You get a page of search results, and the URL for that page looks like this:

http://www.google.com/search?q=google+pac-man

See the part at the end? Embedded as part of the URL, the page’s address, are the search terms you used, “google pac-man.”

Now you click on one of the pages listed in the search results. When you leave Google and arrive at the page you selected, that page gets sent a copy of the URL above. Then, using web analytics software, the page’s owner can easily tell that you found their page by doing a search for “google pac-man” on Google.

Similarly, I have a personal blog called Daggle. From time-to-time, I link from that blog over to my work blog here at Search Engine Land. For example, I wrote a piece on my personal blog in April called Dear Facebook & Google: We Are Not Your Pawns – Enough With The Auto Opt-In!

The URL for that page looks like this:

http://daggle.com/dear-facebook-google-pawns-optin-1796

I linked from that article to a story here at Search Engine Land. If someone clicks from my personal blog to that story, the referring page (my page on my personal blog) is logged, and I know exactly where they came from.

FYI, when I’ve tested in the past, referrer data is only sent if you click on a link from one page to the next. If you visit a page, then type in a new URL directly into your browser to bring up a new page, the “old” address is not sent.

The Web’s Caller ID

Hopefully, you can see why I think of the referrer as the web’s caller ID. But it’s more caller ID for places, rather than individuals.

You know how Caller ID sometimes lists a business name, while other times an individual’s name is shown? It’s all down to how the telephone is registered.

For the most part, referrers do  NOT show the names of individuals. There is other information that web sites record, primarily IP addresses, that potentially can be used to identify who a particular person is. This only works in some special cases. See Google Anonymizing Search Records To Protect Privacy, which goes into more depth about that. Also see here and here about how browser fingerprinting, which is still not that common, can be done to better identify people.

Instead, referrer data is largely designed to show you the last business someone came from. And that information, for site owners, is marketing gold.

I Love The Referrer

When you understand the search terms someone used to reach your site,  you understand how successful your search marketing activities are. You can also better target those people with a message when they arrive.

Aside from search terms, understanding exactly how people are finding your web site tells you what marketing is working or not. You can measure if you’re getting good word of mouth on blogs. If you’ve been mentioned in Google News. If your URL is being passed around on Twitter. If some non-profit web site is linking to your information.

The referrer is what makes internet marketing so measurable, so performance-driver and so unlike traditional marketing, where so little is measured. You know  that quote widely attributed to advertising pioneer John Wanamaker?

Half the money I spend on advertising is wasted; the trouble is I don’t know which half.

If he were alive today, and had done his advertising on the internet, he’d never have been able to say that. Referrers would tell him exactly what’s wasted and what’s not.

Threats To The Referrer

Three things in the past week make me wonder if we’re seeing the beginning of the end for referrer data.

Google Launches Encrypted Web Search from last Friday covers how Google has launched a new way for people to do “secure” searches. In the way you do secure online banking, it means that the searches you do on Google can’t be “overheard” by anyone. Part of this means that no referrer information is passed along.

Protecting Privacy with Referrers out from Facebook yesterday covers how in some limited cases, Facebook referrers could be used to track a visit back to a particular user at Facebook. It’s an excellent look at what a referrer is, by the way. Facebook’s done some fixes, but more important tells us, “Facebook is one site where referrers don’t really belong.”

Google Lets Users Opt Out Of Analytics Tracking, But Doesn’t Expect Many Will from today covers how Google is allowing anyone to opt-out from being tracked by its Google Analytics code. This is code that web site owners install on their web sites to track and understand what people do their sites. Part of that tracking is logging the referrer information.

Love It / Hate It

I love choice. I love that users can have secure searching. It’s something I wanted to see come — though I have to say, I didn’t realize it would be at the cost of referrer information.

I’m also glad that those who want to opt-out of Google Analytics tracking can do so. They’ll still be tracked in plenty of other ways from other companies that don’t offer an opt-out, but people can be especially paranoid about Google. Offering an opt-out is a good PR move by Google, even though it’s individual web sites that use Google Analytics that get hurt the most.

As for Facebook, there’s an argument that Facebook is not the only site where “referrers don’t really belong.” And that leads to a worry about whether other sites will drop them. Plus, I already have a frustration with Facebook. The referrer data it sends, and that Google Analytics parses, is already so poor that it’s difficult to understand exactly how anyone found your content on Facebook.

That’s a big issue for the company. I’ve seen several prominent people suggest that Facebook doesn’t drive traffic. I know first hand that it does, something I’ll cover in a future post. But as a publisher, if you can’t measure Facebook’s impact, it’s easy to write it off. That’s not a position Facebook wants to be in.

Fair To Steal The Referrer?

So, I have mixed feelings about the current trend. Most reassuring on the Google front is that both options, secure search and Google Analytics opt-out, aren’t on by default. Practically no one, relatively speaking, will do a secure search. Practically no one will opt-out of Google Analytics. Taken individually, there’s no need to hit the panic button.

Taken collectively, I’ve got more worries. Add in the Facebook news, and you wonder what’s next. The real death blow would be if Internet Explorer or Firefox or Chrome decided to drop support for sending referral information. That would have a huge impact. Good PR, but terrible for site owners.

Keep in mind that site owners uses referral tracking for other reasons. For example, it can be used to prevent images on your site being loaded on another site without permission. It can be used to help fight spam, to detect if someone’s really on your site when they comment. It can be used to detect what people do within your own site — the paths they take, to determine how usable your site is. And, of course, there’s the marketing component.

I’m not a big fan of those who cry out that the web is full of net neanderthals who just want content for free. But then again, as someone who has published content online for 15 years, I know how difficult it can be to earn off the web. Ad rates are far below where they should be when you consider the time people spend on web sites. Subscription income can be done, but it’s a huge amount of work.

Having referrer data has been a crucial tool for the online publisher. It has felt like a fair trade-off that visitors would provide it. I don’t want to see it go. I don’t know what we’d do if it did go.

Oh, The Irony

My hope is that this all blows over. That blocking referrers remains something only the tech-savvy or the super paranoid do. That blocking referrers remains something that only happens if people jump through hoops with the opt-outs that Google provides. That it only comes to browsers if you use plug-ins like No Referer.

That’s a plug-in I know well, because I use it. Yes, my name is Danny, and I’m a referrer blocker.

I deal with a lot of embargoed information and sites that aren’t yet meant for public viewing. I block to ensure I don’t accidentally “leak” the locations of these sites. And I’m well aware of the irony of writing that I don’t want referrers to go away while at the same time, I personally actively block them.

I don’t have a good answer to that. Maybe I haven’t moved to the acceptance phase of loss. Heck, we haven’t even had the loss of referrers yet.

I can tell you that life in a referrerless world can be hard. I’ve discovered some bank web sites just don’t work. I can’t comment in some places. I can’t get some web sites to function as they should.

When I encounter these issues, I eventually think “Hmm, maybe they need a referrer?” Then I switch those on, for those sites, and things work again.

I suspect we’ll see more of this, if referrer data gets blocked more and more. Sites might not let you in, unless you provide a referrer, similar to how some sites will demand cookies.

We’ll see. Or maybe someone will come up with a clever way for referrer information to continue being provided but also while making things more private.

This post will present a fast and easy way to get your behavioral targeting (BT) process up and running. Although any discussion of BT can sometimes turn into a PhD-level conversation, it is really important to pluck some low-hanging fruit to convince executives that it is a worthwhile journey. If you do, your path will likely be paved with gold.

Recently, BTBuckets, a free behavioral targeting and segmentation tool, launched two brilliant plugins to integrate their tool with Google Website Optimizer and Google Analytics. These free tools are a boon for anyone who seeks to target the highest quality audience to a website.

Targeting users from within Google Analytics

The first plugin, for Google Analytics (GA), enables users to create behavioral targeting campaigns from within GA for a series of reports (see report details in the BTBuckets plugin page).

Say, for example, you post a press release on your corporate blog describing a very important new feature of a key product. Naturally, you get many incoming links from blogs and news websites. However, bbc.co.uk links to your blog home page instead of linking to the feature description post itself. During the time the post is at the top of the blog home everything is great, and your traffic converts pretty well.

However, after you publish a few additional posts, you note on Google Analytics that your conversion rates for visitors from the BBC (which still sends a good amount of traffic) drops drastically. This is most certainly a consequence of visitors not finding what they were looking for. One way to deal with that is to create a redirect for traffic arriving from BBC to the blog home and lead these visitors to the specific post (see explanation on video below). This can now be done from within GA with the help of this plugin.

Click here to view the embedded video.

Test and target using Google Website Optimizer

The second plugin, for Google Website Optimizer, enables users to segment visitors that will be included in a test. It is important to note that the plugin works for specific user segments including new visitors, returning visitors, search and social media.

For example, suppose you are a publisher, and display AdSense advertisements. This is a great way to earn a few bucks out of your website; however, is it a good strategy to show ads to all your visitors? Perhaps, perhaps not. If someone is a first time visitor, it is probably better to show a special promotion, or maybe a banner for an internal campaign, or a signup form for an email subscription or an RSS feed. A returning visitor is more likely to know your brand and is likely to return even if he or she clicks on a contextual ad. For these visitors, instead of showing a signup form, it’s fine to display AdSense ads.

Since you can never know if that is the right thing to do without testing, you should create two tests, one for new visitors and one for returning visitors. One test displays AdSense ads for 50% of visitors, and the other displays an internal banner to the other half. If you happen to find a significant difference, bingo! If not, keep testing and targeting.

Below is a video showing how to use the plugin, and more info about it can be found on BTBuckets plugin page.

Click here to view the embedded video.

Now we have a suite of tools to analyze, test and target for free. Are you doing behavioral targeting? If so, please share your successes and challenges in the comments section below.

The first step to a successful behavioral targeting process is finding the right targets. It is not always obvious which users should be “bucketed” together. Creating effective buckets requires knowledge about the site and a careful analysis of the data. It is important to have clear objectives; this way we can pinpoint the type of person we are targeting, measure our success and optimize for maximum effect.

This can be done using any web analytics tool that provides advanced segmentation. But you also need to think hard about the marketing implications of segmentation and persona creation. I chose to use Google Analytics in this example since it is free and it provides a very powerful segmentation feature.

What exactly is a persona?

According to Wikipedia, “A user persona is a representation of the goals and behavior of a real group of users.” This definition meets the objective of behavioral targeting as provided in my last article: create a unique experience for each visitor. If users have different goals and behaviors when they come to a website, why should they have the same experience?

Web marketing guru Bryan Eisenberg is among the pioneers that introduced personas into the online world as a way to segment users on websites and provide personalized experiences. This technique requires understanding the website objectives and users very deeply. As Bryan notes in some of his writings, personas can be used to optimize websites and target users in pretty much every aspect of online marketing: campaign creation and expansion, competitive analysis, offline advertisement and onsite content targeting.

It is important to differentiate between pure segmentation and persona creation. While segmentation is critical to behavior analysis (or, as Avinash Kaushik says, analyzing data in aggregate is a crime), personas cannot be achieved by clickstream data only. This requires a deep marketing analysis: understanding who is your market and what they want from you. Behavioral targeting takes advantage of both techniques. We can target different segments, such as new visitors, returning buyers, Canadian returning visitors, or any segment that shows a special behavior. And we can target different personas, trying to show different content to people coming with different goals to the website.

Creating personas

Building personas is a deep marketing exercise: you must understand your audience and the product you are offering. Although data should be used (as seen below), the structure should come from the company’s understanding of the market and the customers. Bryan Eisenberg provides a series of helpful questions on measuring personas for success, which can be very useful to get started and build your personas. Below are a few of them:

• What does this persona do on a daily basis?
• What’s the persona’s life mantra?
• What’s this persona’s unspoken question regarding this product?
• What does she expect from this product?
• What information will this persona need to be persuaded to take action?
• Why is she motivated to take this action?
• What actions do you want this persona to take, and how will you persuade her to take them?

You can also take a look at this nicely formatted persona sample (PDF) to get an idea on how to create a document to convince management of the value of this exercise.

Creating your persona using Google Analytics

First of all, it is important to note that it is very difficult to translate a persona into a measurable segment based on clickstream data, but it is possible to reach an approximation. Let’s suppose we want to develop a persona called Danny Sullivan which I believe represents the kind of people that visit my site. I would start by trying to answer the questions above about him, and I might as well try to find a person that has the same lifestyle as Danny to help me (Jim Sterne?). Then I would go about trying to find some metrics and dimensions on Google Analytics that help me pinpoint the persona and turn them into an advanced segment. Here’s an example:

Click for a high resolution image.

As you can see above, I believe that Danny (the persona) has the following behavior patterns:

• Since he is always on the road, he probably visits the site using his smart phone (iPhone, Blackberry or Android)
• Usually in the United States
• He has been to the website before, so he is a returning visitor
• He reads many blogs using feeds, and probably gets to one of my posts through Feedburner (see new integration between Feedburner and Google Analytics)
• Since he comes to my site every day, ‘Days since last visit’ equals 1 or 2
• And since visiting my site is probably the first thing he does in his daily routine, his visit probably happens before breakfast, or at least earlier than 10am.

This is a somewhat simplistic example, but it shows that it is possible to create personas using Google Analytics to understand how each targeted audience is behaving on your site. This shows us what we are succeeding or failing to provide to each kind of person on the website.

I’ll illustrate how we will use the information above to feed the behavioral targeting cycle in my next post.

According to Wikipedia, there are two principal types of behavioral targeting:

Onsite behavioral targeting is a technique that uses visitor behavior to target certain content that is proved to be more relevant to a segment of visitors. It should be preceded by an encompassing analysis of users and their characteristics (using web analytics tools). We can either define hard rules (for example, offer a special deal to anybody that adds any two or more products to a shopping cart) or use an engine to dynamically learn about and then target high-converting groups.

Offsite behavioral targeting is technique used by advertising networks to increase advertisement targeting. For example, Google is using this type of targeting to profile visitors to their website network according to subjects they like (their “interests) and then uses this info to provide users with targeted ads across the entire content network.

In this article I’ll focus on the first type of BT, onsite behavioral targeting, the type of technique used by website owners to improve user experience by delivering the right content to each person. Companies that current provide such a solution to website owners are:

• BTBuckets (free tool)
• Sitebrand
• Amadesa
• Omniture Test&Target
• SiteSpect

From tracking to behavioral targeting

Web analytics has been constantly developing since the 1990s. In the beginning there was data, and initially the struggle was to collect it accurately and provide reports on the state of websites, usually from an IT perspective. Then, with the turn of the millennium, analysts felt the necessity to turn numbers into insights, and the field evolved from simple data reporting to analysis. Today, marketers increasingly understand that testing is the way to go when it comes to design and implementation of websites (in other words, intuition-based decisions don’t really work well). Said another way, the customer should decide what works and what doesn’t (this phase is still rapidly evolving).

In the last two years, marketers, analysts and executives have started to understand that customers should get what they want without having to ask for it. That’s what behavioral targeting is all about: delivering the right content to each visitor to a website. It moves the current focus on overall results to segment results. It enables the website owner to understand which visitors are struggling and which are succeeding with their objectives. It helps marketers build more relevant campaigns to target the right market, be it through search, content, media, or emails. These insights and actions should not come at the expense of website testing, but in addition to it. Testing is very important to recognize the low hanging fruit that is ready to be plucked. It is also a great way to persuade management of the importance of investing in both testing and targeting.

Analytics guru Jim Sterne defined the benefits of behavioral targeting as follows:

“We can attract prospects with customized campaigns according to their interests, engage site visitors with dynamic content in response to their conduct and desires, and put the right message in front of the right person at the right time. We can create a more pleasant and more individual buying experience. We can quickly identify the offers that will more likely convert those prospects to buyers.”

However, the market is still not completely ready for this revolution. Recent research from eMarketer suggests that American internet users are not very fond of behavioral targeting techniques. As seen in the chart below, one of the conclusions of the research is that “respondents showed somewhat more interest in receiving personalized discounts and news, but still, less than one-half of Americans wanted any tailored Web content at all.”

Concluding, we can see a strong trend towards using behavioral targeting to provide web users with richer web experiences. But this will require a market education effort so that users don’t perceive companies using these techniques as not respecting user privacy.

In my next article I will go over a few examples showing how to implement behavioral targeting and analyze its results to increase website conversion rates.

]]> http://searchengineland.com/behavioral-targeting-creating-a-unique-experience-for-each-visitor-30015/feed 1 http://searchengineland.com/is-web-analytics-easy-or-difficult-28098 http://searchengineland.com/is-web-analytics-easy-or-difficult-28098#comments Fri, 30 Oct 2009 10:00:01 +0000 Evan LaPointe http://searchengineland.com/?p=28098 Two rockstars have emerged in the web analytics field to date. We all know who they are: Eric Peterson has demystified analytics for us, while Avinash Kaushik has helped us take it one day, or dare I say, “an hour a day,” at a time. And one of them says web analytics is easy while the other one says web analytics is very hard. So who is right?

Why web analytics is easy

Yes, it’s true. Web analytics is easy, according to Avinash Kaushik. It’s actually incredibly easy. Assuming that the desired outcomes of web analytics are changes that will positively impact your site and your bottom line (what other purpose can you think of?), you’ll find that even complete beginners can drive surprising value.

Web analytics is this easy because your web site has more issues than you’ll see in a month of Dr. Phil episodes. Starting with some of Kaushik’s favorite metrics like bounce rate by landing page, it’s a virtual walk in the park to identify a host of obvious issues with your site, your marketing, your calls to action and more.

Bounce rate / percent of single page view sessions (times where users enter your site and leave almost immediately) is the online version of your significant other saying “hell no” when you walk up with that new striped shirt you thought was so snazzy. Don’t take it personally when people reject your content, especially when you thought it was great. It isn’t. Just get over it and change it for the better. Test it using website optimizer. Your site is there to make you money, not generate pride.

Web analytics is also easy because you can learn as you go. Almost all valuable insights you will come across in your web analytics quest are a result of you saying, “Huh, that’s interesting.” Then you dig a little deeper, find some answers, and come up with a new way of doing things. It’s rare to be unsurprised by your findings when you’re digging through Google Analytics or Omniture SiteCatalyst or any other tool, because what you’re really looking for are things that you don’t expect.

Whether you’re a seasoned veteran or a complete newbie, you’re going to have to dig in and figure out what’s causing that drop in page views per session or conversion rate or video completion rate or whatever else you’re measuring, and it’s almost always going to be a learning experience. The conclusions and efficiencies that come from the seasoned veteran may be superior, but the newbie can definitely hold the fort, much more than we often give them credit for.

Here are some typical problem areas where you can take advantage of “easy” analytics to gain insights and improve your site:

• Landing pages with the highest bounce rates
• Landing pages with the lowest conversion rates
• Referrers with high bounce rates
• Referrers with low conversion rates
• Paid search: keywords with high CTR and low conversion (money fires)
• Natural search: top keywords for major landing pages (are we satisfying those searchers?)

So keep in mind when I say web analytics is easy, my standard is how sophisticated you need to be to drive performance-altering change on your web site. And you don’t need to be that sophisticated to pinpoint a basketful of site issues that need addressing.

But web analytics is hard!

Very hard, according to Eric Peterson. So, web analytics does have a dark side. It can be hard, but also not for the reasons you might expect.

Web analytics is not hard because you need a statistics degree. It’s not hard because you need to make some sort of model that looks like you’re trying to land something on the moon (or blow it up). It’s not hard because you have to be some Excel whiz or know how to run a multivariate test. It’s hard because people believe that analysts are automatically good when they embody these characteristics, which is completely untrue.

The best analysts are good communicators, not NASA engineers. In fact, engineers are usually the most frustrated and unsuccessful analysts out there (in the long run). While they’re trying to explain that the hypotenuse of a thermodynamic econometric assimilator is equal to the perennial habits of in-market visitor segment recency, the clear communicator is explaining that a shift in budget from paid search to an email campaign is expected to generate $2.6 million in incremental revenue. While the NASA analyst is trying to explain this:

… the articulate analyst is in the CFO’s office talking about the tech support and IT prioritizing savings driven by their investment in Tealeaf, in real dollar terms.

This isn’t to say that formulas like this (it looks like Pac Man, math edition) aren’t downright brilliant. They are. They’re just not universally useful in a business environment. They’re very potent research tools that should be used to make complicated data more malleable for the analyst, but they should never see the light of day in the rest of the organization. While the NASA analyst may think this formula will make him look smart and impressive, the executives just see another propellerhead that will waste time with technobabble. No matter how valuable this technobrilliance is, it’s always going to sound like Urkel to the CEO.

Web analytics is very hard, in essence, because we deal with very complex data sets, statistical analysis, trying to tie online and offline data together, seasonality, and more. It takes a very smart person to do this well, without making mistakes. But the most important—and hardest—thing to do is tie it all back to the two very simple metrics that drive all business value: revenue and profitability. This is the language of business, which is different from the language of the analyst. And while the NASA analyst may think these two basic elements of business oversimplified and unsophisticated, the entire history of business demonstrates that these are the only two things that matter.

Something else we can relate to

The same can be said of SEO, for example. Is SEO hard? No, of course not. You can spend an hour on this site and learn enough to make a huge difference. Is SEO easy? No, of course not. Even if you spend 100 hours on this site, you’ll still be making huge mistakes and missing out on nuances that can cost you valuable real estate in the search results. That’s why there are specialists. But don’t let specialists tell you that it’s so hard that you shouldn’t learn the basics and employ simple recommendations that can make a huge difference.

Our industry is rife with smart people who are on border patrol. We tend to get so caught up in how smart and sophisticated our ideas are, we push lay people out and purposefully alienate them with our big words and complicated explanations, telling ourselves that if a CEO doesn’t understand a simple chi-square test, he’s a dumbass. The truth is, we’re the dumbasses when we can’t reduce the outcomes of that test to simple, explainable tactics that will produce either a revenue increase or a cost reduction.

So if you’re the expert, make it simple for people. Make them comfortable. Don’t make it hard. If you’re the beginner, keep it simple, find clear opportunities, ask smart people for advice, and enjoy the ease of web analytics!

]]> http://searchengineland.com/is-web-analytics-easy-or-difficult-28098/feed 6