How To Track Emerging Search Engine Blekko In Web Analytics Systems

More than a year has passed since search engine upstart blekko launched, yet Web marketing analysts using digital media measurement tools like Google Analytics won’t have seen any traffic attributed to blekko in organic search marketing reports. Instead, traffic supplied by blekko will show up in a referring websites report. It is possible to properly […]

Chat with SearchBot

More than a year has passed since search engine upstart blekko launched, yet Web marketing analysts using digital media measurement tools like Google Analytics won’t have seen any traffic attributed to blekko in organic search marketing reports. Instead, traffic supplied by blekko will show up in a referring websites report.

It is possible to properly attribute blekko search traffic referrals by implementing advanced Web analytics configuration techniques.

Is the extra effort worth the trouble? Certainly investors are betting on blekko. In late September, blekko received an addition $30 million investment, half of which was from Russian search engine Yandex.

Why Web Analytics Systems Can’t See Blekko’s Search Keywords

The underlying difficulty many Web analytics systems encounter in recognizing blekko as a search engine is due to a reliance on search engines to specify the search keywords in the request for the destination webpage by using parameters in the referrer URL.

Unfortunately, blekko deviates from what has historically been the de facto standard for transmission of search query information. In the absence of advanced tool configuration, traffic from blekko will show up in a referring sites report instead of an organic search traffic report.

While this scenario is better than nothing, it makes keyword analysis difficult. Knowledge and analysis of search query keywords is a key element of search marketing as a user’s search keywords express their intent when selecting a site from a listing of search results.

In the following discussion, Google Analytics will be used to illustrate solutions, but the same data transformation logic can be applied to other tools as well. The technically inclined are encouraged to read on. Everyone else might want to pass this article on to their technical staff for analysis and implementation.

Deconstructing & “Fixing” The Blekko Search Referrer

A typical blekko referrer URL contains blekko’s domain name, the parameter ws and the user query. Spaces between multiple query words are replaced with plus signs.

In the following example, the user’s search keywords stop+blekko+indexing follow https://blekko.com/ws/ in the search engine results page URL, visible in the browser address bar.

A blekko search result with user keywords in the result URL

Figure 1: A blekko search result with user keywords in the result URL

Unfortunately, this isn’t in the form most Web analytics systems were designed to recognize when processing organic search traffic.

The URL piece ws/ needs to be expressed as a parameter, in the format name = value, i.e. ?ws=stop+blekko+indexing.

Thus: https://blekko.com/ws/stop+blekko+indexing needs to be transformed into https://blekko.com/?ws=stop+blekko+indexing

Two Possible Solutions To Solving The blekko Attribution Problem

There are two potential solutions to proper recognition of blekko as a search engine by Web analytics systems. The first approach is to modify the raw data sent to the Web analytics data collection system. The second is to transform the data collected by the Web analytics system before it is processed for reporting.

Option 1: Modify The Search Referrer Sent To Google Analytics

The blekko referrer URL needs to be transformed into the de facto standard format which includes the user query using URL parameters, e.g. ?query=keywords, a format recognized by most Web analytics systems. JavaScript is well suited to this type of task.

The HTML standard doesn’t allow direct manipulation of the document (page) referrer, but the referrer sent to the Google Analytics data collection system can be changed by assigning the modified value to a special parameter, _setReferrerOverride.

In the asynchronous Google Analytics tracking code which follows, the code in bold will transform blekko referrer URLs and assign the result to _setReferrerOverride.

The code:

  • assigns the page referrer to the variable blekref
  • tests to see if blekref contains the string https://blekko.com/ws/
    • if so, tests to see if https://blekko.com/ws/already contains URL parameters, which start with a ?
      • if so, substitutes the ? with a & which indicates successive URL parameters
    • substitutes ws/ with ?ws=
    • places the modified URL in the Google Analytics variable _setReferrerOverride

<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount','UA-xxxxx-1']); var blekref = document.referrer ; if (~blekref.indexOf('https://blekko.com/ws/')) { if (~blekref.indexOf('?')) { blekref = document.referrer.replace('?','&'); } blekref = blekref.replace('/ws/', '/?ws='); _gaq.push(['_setReferrerOverride',blekref]); } _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'https://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Note: this code will need to be modified if blekko changes their URLs, such as using www.blekko.com in addition to blekko.com.

As with any code changes, do be sure to consult current Google documentation and fully test changes before putting them in production. JavaScript errors in a page can stop successive JavaScript from being executed. No warranties are provided, implied or otherwise!

Tell Google Analytics How To Recognize blekko As A Search Engine

Once blekko’s referrer string has been transformed into the standard query parameter = keywords format, the second and final step is to add blekko to Google Analytics’ list of known search engines.

This also has to be done within the Google Analytics tracking code inserted in each page of the website.

The syntax to add a search engine is of the form:

_gaq.push(['_addOrganic','blekko','ws']);

where blekko is a string in the referrer domain and ws is the query parameter name.

This has to be inserted somewhere before the ['_trackPageview'] call, i.e. from the above example we can change

_gaq.push(['_trackPageview']);

to

_gaq.push(['_addOrganic','blekko','ws'],['_trackPageview']);

A more in depth discussion on working with the _addOrganic configuration option can be found in an article on Google Analytics search engines list customization.

Tracking Page Speed In Google Analytics?

During testing, it emerged that Google Analytics would ignore the value of _setReferrerOverride if _trackPageLoadTime was specified.

Using the Google Analytics debugger, it became clear that Google was deprecating _trackPageLoadTime so the message is simple: remove _trackPageLoadTime from existing tracking code to avoid unforeseen consequences.

Option 2: Transforming Collected Data

Should sites not be able to modify their Google Analytics tracking code, or should their Web analytics system not provide a referral override option, there is a second approach which should work as well. The technique involves transforming the collected data before reports are created.

In Google Analytics, Profile filters perform this task. Filters are a wonderful, but underutilized, Google Analytics feature and this is a great example to show off their potential. The process is:

  1. Identify search referrals from blekko
  2. Update the medium, source and term campaign fields accordingly.

Unfortunately, Google Analytics only allows a profile filter to update one field at a time, so it will be necessary to create multiple filters.

To do this, enter the Google Analytics administration area and create new advanced custom profile filters as illustrated below. Filters can be created at the account level, making it easy to quickly apply the filters to multiple profiles.

Blekko set campaign medium filter.

Figure 2: Blekko set campaign medium filter.

The above filter selects referral traffic from Blekko which contains /ws/ in the URL and changes the Google Analytics Campaign Medium from referral to organic (required for blekko traffic to appear in the organic search report).

Currently, it isn’t strictly necessary to include the Field B requirement, but it may be helpful in order differentiate from paid search should Blekko offer it in the future. Note that the values referral and organic must be in English, even if the Google Analytics interface is in a different language.

Blekko set campaign source filter

Figure 3: Blekko set campaign source filter

Google Analytics will automatically identify blekko.com as the source of traffic, however, some may want to see the source report as “blekko” instead of “blekko.com” in their reports, so as to be consistent with the other search traffic sources. The above filter, which is optional, will do just that.

Blekko set campaign terms filter

Figure 4: Blekko set campaign terms filter

The set campaign terms filter captures the user’s search query and places it in the Campaign Term field, the source of the keywords which appear in the Google Analytics organic search traffic report.

The key is in the regular expression specified in Field A:

^https://blekko\.com/ws/(.+)\??

It says, select referrers which start with https://blekko.com/ws/, capture everything that follows up to a question mark, which may or may not be present, and put the result in the campaign terms field.

Astute readers might note there is still one problem remaining: multiple keywords will still be separated by one or more plus signs which should be converted to spaces.

Unfortunately, it doesn’t seem that Google Analytics supports global (multiple) substitution using filters, but in the spirit of crowd sourcing, a reader may want to chime in to improve this last step.

One work around might be to create multiple substitution filters which will replace a single + with a single space:

Filter to substitute one plus sign, +, in blekko search keywords, with a space

Figure 5: Filter to substitute one plus sign, +, in blekko search keywords, with a space

Once the profile filters are created, they need to be activated by apply them to each Google Analytics profile.

The order in which the filters are applied is not important, with the exception of the plus sign substitution filters, which must come after the set campaign term filter. A Google Analytics best practice is to keep at least one profile without any filters as once data has been transformed by filters as there isn’t any way to revert filter changes.

An unadulterated profile is also helpful in troubleshooting any issues which may arise. New filters can (and should) be tested on a new “throw-away” profile before being applied to production profiles.

A Note About blekko’s Slash Tags

One of Blekko’s distinguishing features is the ability, encouraged by blekko, of users to further restrict their search queries using search operators and filters, what blekko calls slashtags. As the slash tags do condition the user’s query, and to a degree express the user’s intent, neither of the approaches illustrated above attempts to remove slashtags, if present, from the search query keywords.


Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.


About the author

Sean Carlos
Contributor
Sean Carlos is President of Antezeta Web Marketing, based in Milan Italy. As a web marketing teacher and consultant, Sean Carlos helps companies optimize their business results online. He is particularly interested in International SEO, Social Media and Web Analytics.

Get the must-read newsletter for search marketers.