How Twitter Might Send Far More Traffic Than You Think

Over the past year, I’ve seen many people report that Twitter can send tons of traffic to a web site. Certainly I’ve seen first-hand how Twitter has become one of the top non-search referrer sources for Search Engine Land and for some other sites I oversee. But as much as you think Twitter is driving traffic, it might be sending even more that you’re unable to measure.

One key culprit may be that a large number people view web pages using mobile Twitter applications such as Tweetie. Click on a link from a tweet in some of these applications, and they load web pages using their own browsers. Those browser may not run JavaScript. And if JavaScript doesn’t run, then some hosted metrics tools like Google Analytics never realize that your page was viewed.

Consider this recent example. I wrote an article on my personal blog about the collapse of the Clear airport security program last night and how it seems likely that alternatives to it will also go away (TSA Stays Silent As Its Registered Traveler Program Melts Down).

I tweeted that article using Bit.ly, a URL shortener that allows me to track how many people clicked on the link I tweeted:

Bit.ly Stats

The screenshot shows that Bit.ly had seen 343 total clicks to that URL through its system (when I did the screenshot), 339 of them coming from the shortened version I created. So Google Analytics ought to show me roughly that same number of people coming to the page from Twitter, right?

Nope. Google Analytics reports that Twitter sent 63 visits:

Google Analytics Stats

I know some visits from today have yet to be reflected in Google Analytics. That’s why you see the line drop. But I also know that those visits, based on my Bit.ly data, should be about the same as yesterday. In short, at best, Google Analytics is going to report I had about 130 visitors to the article from Twitter. But Bit.ly’s saying I had about 340 visits. Where are the missing 200 or so people? How come Bit.ly’s showing more than three times the visits overall to the page than Google Analytics does. I’ve seen this time-and-time again for various stories, sometimes with gaps of several thousand visits.

This is where the JavaScript stuff I’ve mentioned comes in. It seems to be the best candidate for explaining the discrepancy. Apparently, a lot of people on Twitter use mobile applications, or view from mobile devices, that aren’t processing the Google Analytics tracking code. There are also a few other reasons, as Bit.ly’s general manager Andrew Cohen explained to me in email:

Good question. You’re right about embedded browsers that don’t run JavaScript. Clients that decode bit.ly links but don’t click through to the underlying content are another challenge. If you were to download a browser plug-in to automatically expand short urls, for instance, it would look a lot like a human user. We’d see a GET request from an API, return the underlying long URL, and increment the click count … but we wouldn’t be able to see the actual CTR [clickthrough rate].

Absent JavaScript on the page, it can be tough to distinguish between a decode event and an actual click-through. That’s why we think that bit.ly complements rather than replaces JavaScript-based packages like Google Analytics or Chartbeat. One precaution that we’ve taken, though, is to filter out HEAD requests:

http://blog.bit.ly/post/89178273/talking-heads

We’re also working on some predictive modeling techniques to screen out more bots and scrapers from our click totals.

You might be interested in this article from the MIT Technology Review — http://bit.ly/qrjrf — which shows that these kinds of metrics issues are an industry-wide challenge.

I plan to look at the issue more in the future, but I wanted to get a post out there now with my initial findings and see what others have to say (please comment below!).

Somewhat related, also consider reading this post from Bit.ly: Registered Applications And Better Click Referrer Data.

Just as some referrers might not be seen if JavaScript isn’t loaded, some of the ways people are following tweeted links outside of Twitter aren’t registering, because the applications don’t send any referrer information. So, you’ll see these as “direct” visits as if someone typed in a URL, rather than clicking on a link. But by working with vendors, Bit.ly is trying to improve the picture (other tracking providers may also be doing the same, too).

Related Topics: Channel: Analytics | Features: Analysis | SEM Tools: URL Shorteners | SEM Tools: Web Analytics | Top News | Twitter

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://blog.agrawals.org rakeshlobster

    With all the traffic that mobile Twitter clients are driving these days, it’d be nice to see more sites automatically serving up mobile versions of their pages to these clients.

    My pet peeve is Techmeme which downloads their huge hard to navigate front page on each click.

  • http://www.searchmarketingcommunications.com Cohn

    Thanks for clearing this up Danny…

    Twitter traffic via Bit.ly doesn’t register at all in WordPress hosted blogs stats.

    Also – when posting other content sources in Bit.ly , their stats show all traffic Bit.ly sent to that particular page in addition to your particular bit.ly address.

  • Missmcj

    I believe you entirely that Twitter generates a lot of traffic (or can do), but does it convert? I’ve heard and seen a lot of it being low quality. What have you seen? Would it be possible that your missing 200 went into the Direct Traffic” pool? This is probably because of the APIs being used which I’m sure you know :)

    I’m really interested in social media traffic, especially Twitter because it’s so hard to measure, and also how do you qualify reach? What is the ROI on it? It depends on what your intention is as well of course. Do you want new sign-ups/customers or are you looking to do customer service?

    Liked this post !

  • http://www.positracker.com PosiTracker

    Beyond non tracked traffic, there’s traffic developed from websites that publish twitter feeds…

  • mayank

    I completely agree with the post Danny. A lot of people are realizing the importance of twitter to direct traffic to their website. Techrunch having their own shortend url domain name or recent talk by Fred Wilson at 140conf is a good example. But I think what Andrew Cohen is saying is also quite valid. Bit.ly essentially counts the number of GET’s for the shortened URL. This can be from your browser/twitter client expanding in the background, or loads of crawlers dereferencing the URL to index the target page. I think if you have a huge followers to whom you have tweeted your blog, bit.ly will invariably give a huge hit. The reality however maybe otherwise. Hence the statistics from bit.ly has to be taken with a grain of salt.

  • danmiller

    It looks like you’re basing your data from Google Analytics on referrer, which would also exclude regular browser traffic from third-party tools like TweetDeck.

    You should add tracking parameters to your original URLs before shortening with bit.ly so they’ll appear as campaigns in GA. Or you can use simpler URL parameters if your analytics tool lets you analyze URL parameters directly, like LyrisHQ/ClickTracks (Disclaimer: I work for Lyris).

  • http://www.tysonkirksey.com tysonlee19

    To add to what Dan said, you can use this Firefox extension to automatically add GA tracking parameters to your short URLs.

    https://addons.mozilla.org/en-US/firefox/addon/11492

    (Disclaimer: I wrote this extension, but it’s free, so enjoy!)

  • http://www.highrankings.com/newsletter/ Jill Whalen

    Yep, what danmiller said.

    Danny, you mentioned that you had Google tracking codes, but that’s not going to make it appear that you got referrers from Twitter since hardly anyone reads and clicks links from twitter.com itself.

    What sort of code did you add? If you added campaign tracking codes then go to Traffic Sources > Campaigns and you should find your missing traffic in there.

    @Missmcj you are correct that a lot of Twitter traffic does not convert and simply bounces, however, that depends on a number of things including who you have as followers, what you’re tweeting, where you’re sending them and what action you want the visitor to take (if any).

    If you’re just sending people to an article, most will read it and leave, which is fine. I recently learned that Twitter can indeed convert if there’s an offer involved and the right people are reading your tweets. I was trying to unload the last of my SEO training DVDs at half price and tweeted about it one Sat. morning with the number that were left. I think I had about 5 or 6 left at the time, and within a few hours, they were all sold, all because of a bunch of tweets saying, “okay now only X left, get yours while you can” and that sort of thing!

  • http://www.brickmarketing.com nickstamoulis

    Hi Danny, – This is a very interesting analysis! I too would like to get this code so I can test this as well. It is very interesting to see the quality (and sometimes lack of) of people that I have seen from Twitter…as far as I am concerned, Twitter is another great spot to develop and maintain relationships!

  • http://shig.odaniinteractive.com shig

    I just launched a blog a couple of days ago, which I have promoted only using twitter. Bit.ly showed 6 clicks and GA showed 6 visitors, so they were matching up. I think where the confusion arises is with referrers. Of the 6 visitors, GA reported 1 as coming from twitter and 5 as being direct traffic. This means that one visitor clicked on the link using twitter on the web, and the other 5 were using non-browser twitter clients, which can’t act as referrers unless they register with bit.ly.

  • http://www.clicktrue.biz ginocarpio

    Hi Search Engine Land – Can you give me tips how to get more followers from Twitter?

  • mayank

    My blog post http://zebugroup.com/blog/2009/06/link-tracking-lies-damn-lies-statistics/ addresses the reason why statistics provided by url shortening services may be misleading. This is what I was refering to in an earlier comment.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide