The Death Of Web Analytics? An Ode To The Threatened Referrer

One of the most important online marketing tools is the referrer string. Little known to most web surfers, this is effectively the Caller ID of the internet. It allows web site owners and marketers to know where visitors came from. It’s crucial marketing data, and data that might be going away. What’s The Referrer? When […]

Chat with SearchBot

One of the most important online marketing tools is the referrer string. Little known to most web surfers, this is effectively the Caller ID of the internet. It allows web site owners and marketers to know where visitors came from. It’s crucial marketing data, and data that might be going away.

What’s The Referrer?

When you visit a web page, by default, every major browser (Internet Explorer, Firefox, Chrome…) reports the last page that you viewed before clicking over to the current page you’re viewing. IE, what page “referred” you to the current page. This information is known as the “referrer URL.” (Technically it’s the “referer” string, because of a misspelling in the HTML technical specs years ago).

For example, let’s say that you did a search on Google for “google pac-man,” the game that Google debuted last week. You get a page of search results, and the URL for that page looks like this:

https://www.google.com/search?q=google+pac-man

See the part at the end? Embedded as part of the URL, the page’s address, are the search terms you used, “google pac-man.”

Now you click on one of the pages listed in the search results. When you leave Google and arrive at the page you selected, that page gets sent a copy of the URL above. Then, using web analytics software, the page’s owner can easily tell that you found their page by doing a search for “google pac-man” on Google.

Similarly, I have a personal blog called Daggle. From time-to-time, I link from that blog over to my work blog here at Search Engine Land. For example, I wrote a piece on my personal blog in April called Dear Facebook & Google: We Are Not Your Pawns – Enough With The Auto Opt-In!

The URL for that page looks like this:

https://daggle.com/dear-facebook-google-pawns-optin-1796

I linked from that article to a story here at Search Engine Land. If someone clicks from my personal blog to that story, the referring page (my page on my personal blog) is logged, and I know exactly where they came from.

FYI, when I’ve tested in the past, referrer data is only sent if you click on a link from one page to the next. If you visit a page, then type in a new URL directly into your browser to bring up a new page, the “old” address is not sent.

The Web’s Caller ID

Hopefully, you can see why I think of the referrer as the web’s caller ID. But it’s more caller ID for places, rather than individuals.

You know how Caller ID sometimes lists a business name, while other times an individual’s name is shown? It’s all down to how the telephone is registered.

For the most part, referrers do  NOT show the names of individuals. There is other information that web sites record, primarily IP addresses, that potentially can be used to identify who a particular person is. This only works in some special cases. See Google Anonymizing Search Records To Protect Privacy, which goes into more depth about that. Also see here and here about how browser fingerprinting, which is still not that common, can be done to better identify people.

Instead, referrer data is largely designed to show you the last business someone came from. And that information, for site owners, is marketing gold.

I Love The Referrer

When you understand the search terms someone used to reach your site,  you understand how successful your search marketing activities are. You can also better target those people with a message when they arrive.

Aside from search terms, understanding exactly how people are finding your web site tells you what marketing is working or not. You can measure if you’re getting good word of mouth on blogs. If you’ve been mentioned in Google News. If your URL is being passed around on Twitter. If some non-profit web site is linking to your information.

The referrer is what makes internet marketing so measurable, so performance-driver and so unlike traditional marketing, where so little is measured. You know  that quote widely attributed to advertising pioneer John Wanamaker?

Half the money I spend on advertising is wasted; the trouble is I don’t know which half.

If he were alive today, and had done his advertising on the internet, he’d never have been able to say that. Referrers would tell him exactly what’s wasted and what’s not.

Threats To The Referrer

Three things in the past week make me wonder if we’re seeing the beginning of the end for referrer data.

Google Launches Encrypted Web Search from last Friday covers how Google has launched a new way for people to do “secure” searches. In the way you do secure online banking, it means that the searches you do on Google can’t be “overheard” by anyone. Part of this means that no referrer information is passed along.

Protecting Privacy with Referrers out from Facebook yesterday covers how in some limited cases, Facebook referrers could be used to track a visit back to a particular user at Facebook. It’s an excellent look at what a referrer is, by the way. Facebook’s done some fixes, but more important tells us, “Facebook is one site where referrers don’t really belong.”

Google Lets Users Opt Out Of Analytics Tracking, But Doesn’t Expect Many Will from today covers how Google is allowing anyone to opt-out from being tracked by its Google Analytics code. This is code that web site owners install on their web sites to track and understand what people do their sites. Part of that tracking is logging the referrer information.

Love It / Hate It

I love choice. I love that users can have secure searching. It’s something I wanted to see come — though I have to say, I didn’t realize it would be at the cost of referrer information.

I’m also glad that those who want to opt-out of Google Analytics tracking can do so. They’ll still be tracked in plenty of other ways from other companies that don’t offer an opt-out, but people can be especially paranoid about Google. Offering an opt-out is a good PR move by Google, even though it’s individual web sites that use Google Analytics that get hurt the most.

As for Facebook, there’s an argument that Facebook is not the only site where “referrers don’t really belong.” And that leads to a worry about whether other sites will drop them. Plus, I already have a frustration with Facebook. The referrer data it sends, and that Google Analytics parses, is already so poor that it’s difficult to understand exactly how anyone found your content on Facebook.

That’s a big issue for the company. I’ve seen several prominent people suggest that Facebook doesn’t drive traffic. I know first hand that it does, something I’ll cover in a future post. But as a publisher, if you can’t measure Facebook’s impact, it’s easy to write it off. That’s not a position Facebook wants to be in.

Fair To Steal The Referrer?

So, I have mixed feelings about the current trend. Most reassuring on the Google front is that both options, secure search and Google Analytics opt-out, aren’t on by default. Practically no one, relatively speaking, will do a secure search. Practically no one will opt-out of Google Analytics. Taken individually, there’s no need to hit the panic button.

Taken collectively, I’ve got more worries. Add in the Facebook news, and you wonder what’s next. The real death blow would be if Internet Explorer or Firefox or Chrome decided to drop support for sending referral information. That would have a huge impact. Good PR, but terrible for site owners.

Keep in mind that site owners uses referral tracking for other reasons. For example, it can be used to prevent images on your site being loaded on another site without permission. It can be used to help fight spam, to detect if someone’s really on your site when they comment. It can be used to detect what people do within your own site — the paths they take, to determine how usable your site is. And, of course, there’s the marketing component.

I’m not a big fan of those who cry out that the web is full of net neanderthals who just want content for free. But then again, as someone who has published content online for 15 years, I know how difficult it can be to earn off the web. Ad rates are far below where they should be when you consider the time people spend on web sites. Subscription income can be done, but it’s a huge amount of work.

Having referrer data has been a crucial tool for the online publisher. It has felt like a fair trade-off that visitors would provide it. I don’t want to see it go. I don’t know what we’d do if it did go.

Oh, The Irony

My hope is that this all blows over. That blocking referrers remains something only the tech-savvy or the super paranoid do. That blocking referrers remains something that only happens if people jump through hoops with the opt-outs that Google provides. That it only comes to browsers if you use plug-ins like No Referer.

That’s a plug-in I know well, because I use it. Yes, my name is Danny, and I’m a referrer blocker.

I deal with a lot of embargoed information and sites that aren’t yet meant for public viewing. I block to ensure I don’t accidentally “leak” the locations of these sites. And I’m well aware of the irony of writing that I don’t want referrers to go away while at the same time, I personally actively block them.

I don’t have a good answer to that. Maybe I haven’t moved to the acceptance phase of loss. Heck, we haven’t even had the loss of referrers yet.

I can tell you that life in a referrerless world can be hard. I’ve discovered some bank web sites just don’t work. I can’t comment in some places. I can’t get some web sites to function as they should.

When I encounter these issues, I eventually think “Hmm, maybe they need a referrer?” Then I switch those on, for those sites, and things work again.

I suspect we’ll see more of this, if referrer data gets blocked more and more. Sites might not let you in, unless you provide a referrer, similar to how some sites will demand cookies.

We’ll see. Or maybe someone will come up with a clever way for referrer information to continue being provided but also while making things more private.


Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.


About the author

Danny Sullivan
Contributor
Danny Sullivan was a journalist and analyst who covered the digital and search marketing space from 1996 through 2017. He was also a cofounder of Third Door Media, which publishes Search Engine Land and MarTech, and produces the SMX: Search Marketing Expo and MarTech events. He retired from journalism and Third Door Media in June 2017. You can learn more about him on his personal site & blog He can also be found on Facebook and Twitter.

Get the must-read newsletter for search marketers.