The Death Of Web Analytics? An Ode To The Threatened Referrer

One of the most important online marketing tools is the referrer string. Little known to most web surfers, this is effectively the Caller ID of the internet. It allows web site owners and marketers to know where visitors came from. It’s crucial marketing data, and data that might be going away.

What’s The Referrer?

When you visit a web page, by default, every major browser (Internet Explorer, Firefox, Chrome…) reports the last page that you viewed before clicking over to the current page you’re viewing. IE, what page “referred” you to the current page. This information is known as the “referrer URL.” (Technically it’s the “referer” string, because of a misspelling in the HTML technical specs years ago).

For example, let’s say that you did a search on Google for “google pac-man,” the game that Google debuted last week. You get a page of search results, and the URL for that page looks like this:

http://www.google.com/search?q=google+pac-man

See the part at the end? Embedded as part of the URL, the page’s address, are the search terms you used, “google pac-man.”

Now you click on one of the pages listed in the search results. When you leave Google and arrive at the page you selected, that page gets sent a copy of the URL above. Then, using web analytics software, the page’s owner can easily tell that you found their page by doing a search for “google pac-man” on Google.

Similarly, I have a personal blog called Daggle. From time-to-time, I link from that blog over to my work blog here at Search Engine Land. For example, I wrote a piece on my personal blog in April called Dear Facebook & Google: We Are Not Your Pawns – Enough With The Auto Opt-In!

The URL for that page looks like this:

http://daggle.com/dear-facebook-google-pawns-optin-1796

I linked from that article to a story here at Search Engine Land. If someone clicks from my personal blog to that story, the referring page (my page on my personal blog) is logged, and I know exactly where they came from.

FYI, when I’ve tested in the past, referrer data is only sent if you click on a link from one page to the next. If you visit a page, then type in a new URL directly into your browser to bring up a new page, the “old” address is not sent.

The Web’s Caller ID

Hopefully, you can see why I think of the referrer as the web’s caller ID. But it’s more caller ID for places, rather than individuals.

You know how Caller ID sometimes lists a business name, while other times an individual’s name is shown? It’s all down to how the telephone is registered.

For the most part, referrers do  NOT show the names of individuals. There is other information that web sites record, primarily IP addresses, that potentially can be used to identify who a particular person is. This only works in some special cases. See Google Anonymizing Search Records To Protect Privacy, which goes into more depth about that. Also see here and here about how browser fingerprinting, which is still not that common, can be done to better identify people.

Instead, referrer data is largely designed to show you the last business someone came from. And that information, for site owners, is marketing gold.

I Love The Referrer

When you understand the search terms someone used to reach your site,  you understand how successful your search marketing activities are. You can also better target those people with a message when they arrive.

Aside from search terms, understanding exactly how people are finding your web site tells you what marketing is working or not. You can measure if you’re getting good word of mouth on blogs. If you’ve been mentioned in Google News. If your URL is being passed around on Twitter. If some non-profit web site is linking to your information.

The referrer is what makes internet marketing so measurable, so performance-driver and so unlike traditional marketing, where so little is measured. You know  that quote widely attributed to advertising pioneer John Wanamaker?

Half the money I spend on advertising is wasted; the trouble is I don’t know which half.

If he were alive today, and had done his advertising on the internet, he’d never have been able to say that. Referrers would tell him exactly what’s wasted and what’s not.

Threats To The Referrer

Three things in the past week make me wonder if we’re seeing the beginning of the end for referrer data.

Google Launches Encrypted Web Search from last Friday covers how Google has launched a new way for people to do “secure” searches. In the way you do secure online banking, it means that the searches you do on Google can’t be “overheard” by anyone. Part of this means that no referrer information is passed along.

Protecting Privacy with Referrers out from Facebook yesterday covers how in some limited cases, Facebook referrers could be used to track a visit back to a particular user at Facebook. It’s an excellent look at what a referrer is, by the way. Facebook’s done some fixes, but more important tells us, “Facebook is one site where referrers don’t really belong.”

Google Lets Users Opt Out Of Analytics Tracking, But Doesn’t Expect Many Will from today covers how Google is allowing anyone to opt-out from being tracked by its Google Analytics code. This is code that web site owners install on their web sites to track and understand what people do their sites. Part of that tracking is logging the referrer information.

Love It / Hate It

I love choice. I love that users can have secure searching. It’s something I wanted to see come — though I have to say, I didn’t realize it would be at the cost of referrer information.

I’m also glad that those who want to opt-out of Google Analytics tracking can do so. They’ll still be tracked in plenty of other ways from other companies that don’t offer an opt-out, but people can be especially paranoid about Google. Offering an opt-out is a good PR move by Google, even though it’s individual web sites that use Google Analytics that get hurt the most.

As for Facebook, there’s an argument that Facebook is not the only site where “referrers don’t really belong.” And that leads to a worry about whether other sites will drop them. Plus, I already have a frustration with Facebook. The referrer data it sends, and that Google Analytics parses, is already so poor that it’s difficult to understand exactly how anyone found your content on Facebook.

That’s a big issue for the company. I’ve seen several prominent people suggest that Facebook doesn’t drive traffic. I know first hand that it does, something I’ll cover in a future post. But as a publisher, if you can’t measure Facebook’s impact, it’s easy to write it off. That’s not a position Facebook wants to be in.

Fair To Steal The Referrer?

So, I have mixed feelings about the current trend. Most reassuring on the Google front is that both options, secure search and Google Analytics opt-out, aren’t on by default. Practically no one, relatively speaking, will do a secure search. Practically no one will opt-out of Google Analytics. Taken individually, there’s no need to hit the panic button.

Taken collectively, I’ve got more worries. Add in the Facebook news, and you wonder what’s next. The real death blow would be if Internet Explorer or Firefox or Chrome decided to drop support for sending referral information. That would have a huge impact. Good PR, but terrible for site owners.

Keep in mind that site owners uses referral tracking for other reasons. For example, it can be used to prevent images on your site being loaded on another site without permission. It can be used to help fight spam, to detect if someone’s really on your site when they comment. It can be used to detect what people do within your own site — the paths they take, to determine how usable your site is. And, of course, there’s the marketing component.

I’m not a big fan of those who cry out that the web is full of net neanderthals who just want content for free. But then again, as someone who has published content online for 15 years, I know how difficult it can be to earn off the web. Ad rates are far below where they should be when you consider the time people spend on web sites. Subscription income can be done, but it’s a huge amount of work.

Having referrer data has been a crucial tool for the online publisher. It has felt like a fair trade-off that visitors would provide it. I don’t want to see it go. I don’t know what we’d do if it did go.

Oh, The Irony

My hope is that this all blows over. That blocking referrers remains something only the tech-savvy or the super paranoid do. That blocking referrers remains something that only happens if people jump through hoops with the opt-outs that Google provides. That it only comes to browsers if you use plug-ins like No Referer.

That’s a plug-in I know well, because I use it. Yes, my name is Danny, and I’m a referrer blocker.

I deal with a lot of embargoed information and sites that aren’t yet meant for public viewing. I block to ensure I don’t accidentally “leak” the locations of these sites. And I’m well aware of the irony of writing that I don’t want referrers to go away while at the same time, I personally actively block them.

I don’t have a good answer to that. Maybe I haven’t moved to the acceptance phase of loss. Heck, we haven’t even had the loss of referrers yet.

I can tell you that life in a referrerless world can be hard. I’ve discovered some bank web sites just don’t work. I can’t comment in some places. I can’t get some web sites to function as they should.

When I encounter these issues, I eventually think “Hmm, maybe they need a referrer?” Then I switch those on, for those sites, and things work again.

I suspect we’ll see more of this, if referrer data gets blocked more and more. Sites might not let you in, unless you provide a referrer, similar to how some sites will demand cookies.

We’ll see. Or maybe someone will come up with a clever way for referrer information to continue being provided but also while making things more private.

Related Topics: Channel: Analytics | Features: Analysis | Firefox | Legal: Privacy | SEM Tools: Web Analytics | Top News

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://blog.agrawals.org rakeshlobster

    I’m surprised to see that both Firefox’s and Chrome’s private surfing modes still pass referrers.

    That seems to me to be a reasonable compromise vs. stripping referrers entirely from the browser. If someone goes into private mode, referrers are blocked along with history, cookies, etc.

  • gluejar

    I also am a fan of referrer data, But I’ve come to believe that the Referer header is too dangerous not to abolish. It causes web browsing to do things that even web developers don’t expect.

    Last week, I discovered that a firefox add-on developed by bit.ly has the effect of sending them essentially all of my we browsing activity via the referrer header. That’s a horrific leak of my privacy. Full story at http://go-to-hellman.blogspot.com/2010/05/bitly-preview-add-on-leaks-user.html

  • http://phonemarketinginsider.com rbrody

    This article spurred my thinking, that while we see these complications and restrictions for the referrer, we are also witnessing all sorts of new tracking technologies for offline conversions, such as ad tracking analytics and dynamic toll free numbers on websites (which I write a bunch about on blog – Phone Marketing Insider- http://phonemarketinginsider.com). Perhaps people won’t have as many privacy concerns with these specific examples. For example, call tracking really only manifests itself at the time of an offline conversion, at which point people are already more comfortable with the fact that the business they called wants to know where they are coming from, as they are expressing interest in a relationship with this business. What do you think?

  • http://blog.protocol80.com Josh Curcio

    Danny, In relation to this, what are your thoughts on tracking your referrals and whether or not analytics might block this as well. While extra steps need to be taken to track a click from your site to an outbound link, it is not difficult and can create the same privacy concerns as referrals to your site. The ability to track visitors following outbound links will also be simplified in HTML5 with the Ping attribute. More marketers will likely take advantage of these to track things like click-throughs to affiliates, subsidiaries, members of organizations, etc. While they will be looking mostly at data, there are certainly instances where privacy would be a concern.

  • http://scottedwardcummings.com/games/flash-linear-audio-envelopes/ scott_c

    A lot of traffic to the site I work on comes from groups such as Elementary schools and libraries that are traditionally very protective of their computer user’s privacy. I’m worried we’ll take a hit on our stats once the Google Analytics opt-out becomes well known in those communities.

    I expect that this could hit other sites with users concerned with privacy (Slashdot.org, etc) as well.

  • http://searchengineland.com/ Danny Sullivan

    Gluejar, that is scary — thanks for the link.

    Josh, the issue is much less on site referral tracking. You can do that in other ways. It’s about understanding how people came to you. You can’t see the clicks people make on other sites except through the referrer.

  • http://www.alanmitchell.com.au alanmitchell

    Hi Danny,

    I agree – SSL will make analytics pretty much useless, and argue the point in a recent article I also wrote on the subject.

    http://www.alanmitchell.com.au/discussion/google-ssl-page-how-privacy-leads-to-higher-prices/

    I think such a move by Google is a natural progression of the increased fashionability of privacy. We need to realise that as long as data collection is anonymous and not personally identifiable, it’s not actually a bad thing, and is essential for a healthy online economy. Without it, the internet would not be as we know it.

    Cheers,
    Alan

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide