Should we use dashes or underscores in our URLs?

Google re-opened the discussion recently with a new YouTube video that highlights how Google currently handles dashes and underscores, and also reminds us that Google and Bing differ on this bit of SEO minutiae.

Many of you will recall this was a pretty hot topic a few years ago when Google originally announced that both dashes and underscores both served as separators, and then backtracked on that statement about a week later, saying it “wasn’t a done deal yet.”

As the new video explains, it’s still not a done deal … and may never be. Google’s Matt Cutts says there’s still a difference between how Google treats underscores in URLs versus how it treats dashes.

In the video, Cutts explains that, when Google sees an underscore in a URL, it joins what’s before and after into one term. “We still join on the underscore and separate on the dash,” he says.

Perhaps most importantly from an SEO perspective, though, Cutts cautions webmasters against rewriting all of their existing URLs just to have dashes rather than underscores because the actual ranking impact is minimal:

It doesn’t make that much difference. It’s what we call a second-order effect. It’s not a primary thing that makes a huge difference.

And that can be considered a best practice, at least where Google is concerned, for the foreseeable future. In the video, Cutts says Google doesn’t have any employees working on changing how it handles underscores and dashes in URLs.

What About Bing & Underscores/Dashes?

In short: It makes no difference to Bing whether you use underscores or dashes in your URLs. A Bing spokesperson confirmed this for us via email today:

We do not differentiate between dash and underscore in our URL ranking features.

That’s the same response we got from Microsoft back in 2007, when Bing was known as Live Search.

So, What Should Webmasters Do?

The same advice that has applied for years still applies today: If you’re just starting on a new website, use dashes if you plan to place keywords in your URLs. Those keywords might provide a minor signal of what the page is about and help a wee bit with rankings. But if you have an existing website that’s already doing well in Google and Bing — pages are indexed, you’re getting quality natural search traffic, etc. — don’t switch from underscore-based URLs to dashes. The potential problems from changing URLs might be worse than the potential gains from having dashes rather than underscores in your URLs.

The New Domain Names Are Coming

ICANN is the Internet Corporation For Assigned Names And Numbers, and it oversees the world’s domain name system. In its wisdom, it decided yesterday that the 22 generic top level domain (gTLD) names  we have at the moment — things like .com, .org and .net — aren’t enough. More are needed.

ICANN doesn’t have anything up in its news section or its blog on this yet, but within its press room area, there’s a news release (PDF format) with more details, which in turn leads to a helpful fact sheet (also PDF).

Winners & Losers

For $185,000, businesses or organizations can apply for to run their own gTLD. If Apple, Google or Facebook wanted to have .apple, .google and .facebook names, that now becomes possible. That could be pretty cool for companies that want to have domain names that fully reflect their own brands.

Of course, not everyone will necessarily be granted a top level domain name. Who gets lucky depends on what ICANN decides. Smaller brands — which are brands nonetheless — won’t be able to afford the names. Who gets to have hot generic names like .money or .tickets will be decided, to my understanding, solely by ICANN.

What happens when two companies with the same trademark both decide they want the same top level domain remains to be seen. Who gets to be .giants — the San Francisco Giants baseball team or the New York Giants football team. Maybe they could play each other. And those are the only organizations out there with a claim to the Giants name, right?

Domain Names & SEO

In terms of search engines and SEO, so far, I haven’t seen ICANN itself say anything directly about how the new names might be helpful. But my experience in watching the release of longer 63-character domain names back in in 1999, as well as the release new names in the 2000s such as .biz and .mobi, tells me that others will soon be making all types of SEO claims about the new names.

Bottom line — the new names will almost certainly mean nothing special to search engines. They won’t have any super ranking powers. If you managed to get .money, that doesn’t mean you’ll rank tops for money-related terms any more than people with the existing .travel domains do well for travel — because they don’t.

Go do a search for “travel” now or any popular travel-related term on Google. Count the number of times you see sites coming up with .travel in their domain name. You won’t need more than one hand. You probably one need more than one finger. You probably won’t need any fingers at all.

Search engines like Google and Bing give no particular credit or boost to generic top level domain names in general. They don’t say, “Hmm, .com — that’s more important than .net, give it a boost.” They don’t say “Hmm, .travel, boost any site with that over other travel sites.”

Search engines do use country-specific domain names, when these can be trusted, as a signal for making content tailored to a particular country. Many UK web sites use the .uk domain. It’s a trust worthy signal.

In contrast, many non-Libyan or non-Tuvalu businesses make use of the .ly and .tv domain names intended for those countries. As a result, those are a less-trustworthy signal of whether a site is from those countries. That’s why other signals such as links have to be used.

So the new domain names? Sites will do well with them not because they have a tasty top level domain like “.travel” but because particular sites might get enough links and other signals pointing at them to do well.

In the end, the domain names do present new opportunities for some businesses. A few companies are going to get very rich off of this. Some are going to wonder if they need to buy their names again with all these new spaces to avoid cybersquatting.

But from an SEO perspective, be calm. Having a new name won’t rocket you to success; not having one doesn’t doom to you never being found.

To learn more about SEO and ranking signals, see our guides below:

• The Periodic Table Of SEO Ranking Factors
• Search Engine Land’s Periodic Table Guide To SEO

Twitter explained:

When this is rolled out more broadly to users this summer, all links shared on Twitter.com or third-party apps will be wrapped with a t.co URL. A really long link such as http://www.amazon.com/Delivering-Happiness-Profits-Passion-Purpose/dp/0446563048 might be wrapped as http://t.co/DRo0trj for display on SMS, but it could be displayed to web or application users as amazon.com/Delivering- or as the whole URL or page title. Ultimately, we want to display links in a way that removes the obscurity of shortened link and lets you know where a link will take you.

There is also an advertising reason for this:

In addition to a better user experience and increased safety, routing links through this service will eventually contribute to the metrics behind our Promoted Tweets platform and provide an important quality signal for our Resonance algorithm—the way we determine if a Tweet is relevant and interesting to users. We are also looking to provide services that make use of this data, an example would be analytics within our eventual commercial accounts service.

For more information, see the Twitter Blog. Also see our related article, Twitter Gets Its Own URL Shortener To Stop Scams; Good Marketers Need Not Fear.

The news came out during the Duplicate Content: The Search Engines Edition session at SMX East conference today. Google made the announcement in response to site owners again voicing that they wanted such a solution.

The existing canonical tag is only supported by Google, at the moment. Yahoo and Bing both said they’re studying support but think its likely they’ll add support by the end of the year. However, they’ll only support canonicalization across the same domain.

In a way, they’ll support Canonical Tag 1.0 by the end of the year, while Google will be supporting Canonical Tag 2.0, with cross-domain support.

We will have more information when more details come to us.

A bunch of TLDs do not publish expiration dates — how could we compare domains with expiration dates to domains without that information? It seems that would be pretty hard, and likely not worth the trouble. Even when we do have that data, what would it tell us when comparing sites that are otherwise equivalent? A year (the minimum duration, as far as I know) is pretty long in internet-time :-).

But let’s look at some more evidence. Earlier this year, Danny spoke with Google’s Matt Cutts about a variety of domain/link/SEO issues. In light of the claims from domain registrars that longer domain registrations are good for SEO, Danny specifically asked “Does Domain Registration Length Matter?” Matt’s reply:

To the best of my knowledge, no search engine has ever confirmed that they use length-of-registration as a factor in scoring. If a company is asserting that as a fact, that would be troubling.

But wait, there’s more! Shortly after the Q&A with Danny that we posted here, Matt published more thoughts on the matter in a video on the Google Webmaster Central Channel on YouTube.

If you don’t have time to watch the video, Matt says, “My short answer is not to worry very much about that [the number of years a domain is registered], not very much at all.” He reiterates that the domain registrar claims “are not based on anything we said,” and talks about a Google “historical data” patent that may or may not be part of Google’s algorithm. He sums it up by saying, “make great content, don’t worry nearly as much about how many years your domain is registered.”

So we have, essentially, three recent Google statements about the length of a domain registration and its impact on search rankings. None of them specifically say, “No, it doesn’t matter at all.” John’s comment that it would be “pretty hard” for Google to look at inconsistent domain registration data is funny; this is the company that uses complex mathematical equations as a casual recruiting tool:

Anyway, back to the subject at hand. Ultimately, as with many SEO issues, you and I have to decide what we agree with and what we don’t. Google isn’t specifically saying domain registration length doesn’t matter, just that it’s not all that important in the Big Picture. Several years ago, I interviewed Jon Glick, a former member of the Yahoo search team, and he said the length of a registration does matter, but it’s just one signal, one potential flag.

And that’s what SEO is all about, isn’t it? Knowing which flags matter and when. My gut feeling, based on what the Googlers have said, is that if you build a great web site that attracts lots of people, lots of links, lots of attention, etc., it won’t matter if your domain is only registered for one year. But if you build an iffy web site with iffy content and questionable links — in other words, if you look like part of the low-quality, spammy Internet neighborhood — a one-year registration just might matter. Individual SEO factors don’t exist in a vacuum; you can’t easily look at one factor and say it always matters to the same degree for every web site or web page. If CNN.com has a one-year domain registration, it doesn’t matter. If Jimmys-Vegas-Casino-Secrets.com has a one-year registration, it might … depending on what else Jimmy has been doing.

http://www.yourdomain.com/2009/03/19/article-title/

it is less likely to get clicked than the following one, which is the same but without the dates:

http://www.yourdomain.com/article-title/

As I researched this, I noticed that Search Engine Land does not use dates in its URLs, nor does Google spam expert Matt Cutts, nor does Aaron Wall.

Since I have been using dates in my URLs on my blog for over two years, I researched how I could go about changing to not using dates. I found the following code and put it in my .htaccess file:

RedirectMatch 301 /([0-9]+)/([0-9]+)/([0-9]+)/(.*)
$ http://www.domain.com/$4

(Notes: This should be all one line, and to make this work for my blog I had to change “domain.com” to “keenerliving.com”)

Before uploading the updated .htaccess file to my site, though, I had to change the permalink structure in WordPress so it no longer generates URLs with dates in the title. Hence, I modified

/%year%/%monthnum%/%day%/%postname%/

to the following:

/%postname%/

I saved that change, saved my modified .htaccess file to the root directory of my domain, and then re-ran my plugin that generates a sitemap.xml file. Then, I immediately tested the change using several of my older links, to ensure that they redirected properly. I also made sure to activate Alex King’s 404 plugin, which emails me whenever a “not-found (404)” error is generated on my site. It is always a good to activate this plugin whenever you make any structural changes to your blog, as it will help identify any errors you may have created.

It will take a while for Google, MSN, and Yahoo! to reindex these links, but the end result should be a little bit better click-through rate for my domain.

For a while, though, one pays the price of losing page rank for individual pages. At least, that has been my experience … I am told that a 301 redirect should not drop page rank (PR), but somehow it has happened with me. In my case, these changes were made almost 4 weeks ago, and individual page ranks have still not gotten caught back up. A few of my most popular pages that had a Google Toolbar PR of 4 or 5, are currently showing a PR of 0. This may be normal, or it may be that I have done other things that are making it take longer for the PR to catch up (I’d love to hear of your experiences with this in the comments, on what is a normal catch-up time).

It turns out that I did two other things that may be impacting the catch-up time for individual page PR. One is that I went through all 700 of my blog posts and changed every internal link so that the dates were removed. That is, if I had one post that linked to another post using the format http://www.domain.com/year/month/day/linked-post-name/, I changed it link to http://www.domain.com/linked-post-name/. I thought this might speed the re-indexing process up. But I now wonder if this was stupid, and worked against me, or if it was at the very least a waste of time. It is also noteworthy that my blog has a relatively small number of external backlinks (I have not gone after them, although I know I should). There is good internal linking, but the relatively small number of external backlinks could be a factor in the PR not readjusting as one would expect. This is just speculation on my part, but I thought it should be pointed out.

Why go through 700 posts? I also wanted to clean my blog out. In looking at my Google Analytics, I noticed that several people were landing on pages that really should not even exist, and they were bouncing away immediately because the posts were worthless to them. These pages were ones in which I had noted something about a design change, or had apologized for the performance of a server I used a year ago, or some other site news. These posts were semi-relevant for my regular readers at the time the posts were written, but the posts are now worthless. In fact, they are worse than worthless because they have some negative impact on my site’s bounce rate (not a lot, but I am working on improving bounce rate, so I wanted the posts gone).

I did the proper things with these deleted posts, such as adding 410 redirects to my .htaccess file:

Redirect 410 /a-couple-of-site-changes/
Redirect 410 /admin-note/
Redirect 410 /miscellaneous-admin-information/
Redirect 410 /your-input-appreciated/
Redirect 410 /please-ensure-feeds-link-updated/
Redirect 410 /boredom-strikes-again/
Redirect 410 /revised-posting-schedule/
Redirect 410 /what-are-your-thoughts-on-this/
Redirect 410 /implemented-security-update-sorry-for-any-hiccup/
Redirect 410 /get-a-free-issue-of-smartphone-magazine-from-me/

I also updated my sitemap.xml file (so none of the deleted posts were in it), and used the Remove URLs feature in the Google Webmaster Tools to remove each of these deleted links from the Google Index.

But, did making all these changes at once work against me? I cannot say for sure at this time. However, I have learned something from this. Actually, I have re-learned, as I should have known already: when making changes, avoid making too many changes at once. Make one change, and give it time to have its impact (testing it as much as you can), before making other changes.

I hope this is a helpful case study for those of you considering changes in your site. I look forward to your comments.

Update: Two days after drafting this post, the Google Toolbar PR for each of my popular pages (all linked from my sidebar) now have a PR of 4, which is basically what they had before. I do not know if this is due to the redirects finally having their full effect, or whether it is due to the very recent Google PR update, or both.

Postscript: Almost exactly 4 weeks after implementing the redirect shown above, a friend came across a better redirect rule. He uses a slightly different URL structure, of the form site/year/month/postname/ instead of site/year/month/day/postname/. We learned that, for sites with his URL structure, the rule to redirect to site/postname/ is

RedirectMatch permanent ^/[0-9]{4}/[0-9]{2}/([a-z0-9\-/]+) http://example.com/$1

For sites with URL structure site/year/month/day/postname/, the correct rule is

RedirectMatch permanent ^/[0-9]{4}/[0-9]{2}/[0-9]{2}/([a-z0-9\-/]+) http://example.com/$1

Note that these rules apply for blogs that are installed in the site’s root directory. If WordPress is installed in a directory called blog, so that the URLs are of the form site/blog/year/month … , then the correct redirect rule is the same as above but with a change at the beginning: RedirectMatch permanent /blog/[0-9]{4}/[0-9]{2}/ …. (as in the previous two code blocks).

Also, it is important that you place the redirect rule before the redirect rules that WordPress has placed in your .htacess file. The WordPress redirect rules will typically look something like the following:

# BEGIN WordPress

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

# END WordPress

Finally, these redirect rules work on Apache-based servers. If your server is based on Microsoft’s IIS, a different approach would be likely be needed.

The URL Shortener Mega Chart

NOTE (March 2011): I’m hoping to update this article in the near future. Much of the basics are still valid. But you should also see Google URL Shortener Opened To The Public; Comparing To Bit.ly & Twitter, which is sort of an interim update.

To make recommendations, I reviewed various services and how they stacked up in a variety of features. All this got dumped into a spreadsheet, below. You can also view it in full-screen here.

Each column represents a feature. Want to know more about that feature and why it is important? Each is explained below the chart. Green indicates that a particular service gets a good grade for that feature. Red indicates that it is lacking. Not all features are of equal importance, however, so don’t let the occasional red mark make you think a service is lacking. But in general, the more green, the better.

To busy to read the chart or the explanations? There’s a summary of recommended tools at the end of this story. But first, the chart:

301 Redirect

The first column in the chart is for “Redirect.” This is because a top issue to me, and many others, is that a URL shortening service does a “301 redirect” to the full URL. That number stands for the code a web server issues to a browser (or search engine) when a URL is requested.

A 301 redirect says that the URL requested (the short URL) has “permanently” moved to the long address. Since it’s a permanent redirect, search engines finding links to the short URLs will credit all those links to the long URL (see the SEO: Redirects & Moving Sites section of the Search Engine Land members library for more about redirection).

In contrast, a 302 redirect is a “temporary” one. If that’s issued, search engines assume that the short URL is the “real” URL and just temporarily being pointed elsewhere. That means link credit does not get passed on to the long URL.

In short, if you’re hoping that links you tweet will generate link credit for your web site, you want a service that issues a 301 redirect. Also keep in mind that while 301s might be issued today, a shortening service could shift to 302 directs at any time (and if they do, I hope scorn gets poured upon them).

Do tweeted links really matter that much? I mean, it’s not like search engines are really finding these URLs, right?

Wrong. Consider this search on Google:

“http://snurl.com/eom0d”

That brings back 27 links to a story that I twittered last week. Lots of those links are on Twitter pages themselves, from when other people retweeted my link. But some come from FriendFeed, and still others come from places like BusinessWeek, which embeds links from Twitter for various topics.

The bottom line is that when you tweet a link, you have no idea where or how it will spread — but if it does spread, why not ensure your site gets the right credit for it? So get a 301 redirect!

On the chart, you can see which services do 301 redirects (good — thus colored green) versus 302 redirects (bad — thus colored red). One service (Twurl/Tweetburner) issues a 303 code. I have no idea how search engines treat this, but it’s so unusual that I’ve flagged it as red (Google’s Matt Cutts told me he thinks Google may treat 303s as 302s).

Finally, want to test things for yourself? Rex Swain’s long-standing HTTP Viewer is an excellent tool. Enter the short URL into the URL box, untick the “Auto-Follow” box and submit. Then look to see what code is reported.

Tracking

Have people been clicking on your URLs that you tweet? A service like Google Analytics can show you traffic that comes from Twitter, but you might want to drill down further than it allows. A number of URL shortening services provide tracking stats, which in turn range from basic to extensive reporting.

A future article will look at the type of tracking involved. But all things being equal, it makes sense to select a service that provides tracking. Why not have the data if you want it?

To me, tracking is an important feature. That’s why services that offer it get a green for good on the chart and those lacking it a red for bad.

Stability

Nothing is more annoying than tweeting a link using a URL shortener and then having people tweet back at you that the link isn’t working, because the URL shortening service has gone down.

Unfortunately, I don’t know of any regular metrics of which service is the most stable, so there’s no stability column in the mega chart above. My assumption is that those gaining funding and partnerships are more likely to solve capacity problems than others.

Aside from short-term stability issues, there’s also a long-term consideration. What happens if a service shuts down, as happened to Zi.ma? If a service goes down permanently, it takes down all those links that were passing along credit to your site with it. With Zi.ma, fortunately Kl.am was able to keep those links working. But they could have been gone forever.

TinyURL has been around since 2002, so it has some stature in the space. Newer service Bit.ly recently raised $2 million in funding, which suggests it has some people willing to sink money into its future. But neither stature or investment is a guarantee of long-term success.

Twitter Client Support

In lieu of good stability stats, my own preference is to fallback toward depending on services with a lot of use or which are built into Twitter clients. These receive lots of traffic, which should translate into someone deciding they’re valuable, even if the exact revenue models are still being worked out.

Twitstat is one service that reports client metrics (hat-tip to Mashable on spotting that resource). The top current top clients:

• Twitter [web]: 28%
• Tweetdeck: 13%
• Twitterfeed: 6%
• Twhirl: 5%
• Tweetie: 5%

So which clients provide support for which URL shorteners? The mega chart summarizes this above. Any service that’s a default in one or more clients is colored green for good. Red indicates they have no partnerships (or are losing their TweetDeck one).

Twitter itself is used as a “client” by many people to publish their Tweets. If you give it a URL of 30 characters or longer, Twitter will shorten it using TinyURL. Unfortunately, Twitter doesn’t allow for this default choice to be changed. I hope that will happen in the future. Until then, you can always use another service to manually shorten your URLs.

TweetDeck is a nice desktop client (see How To Track Keyword-Based Tweets Within Your Twitter Stream for more about it). Currently, it supports 13 different URL shorteners (oddly including zi.ma, which no longer operates). However, the company recently announced that support for most of these would be removed, with these five continuing or being added.

The mega chart shows those that are going away (flagged as “going”), those that are staying (these just say “TweetDeck”) and the one about to be added (flagged “coming.”) Bit.ly is the default choice, so it’s noted that way.

Twhirl is another desktop client that I personally use, and it’s fourth on the popularity list above. The latest version for uses the new DiggBar service by default. This is not recommended for use, as explained further below.

Twitterfeed isn’t really a client. It’s designed to automatically feed blog posts into a Twitter account. However, you can choose from a variety of URL shorteners rather than go with the default choice of TinyURL. Changing to a service that provides tracking seems a wise choice.

Tweetie is an excellent Twitter client for the iPhone. Unfortunately, it offers no URL shortening support (and since the iPhone absurdly doesn’t offer copy-and-paste yet, there would still be issues even if it were offered). However, in comments from my survey below, someone said that Bit.ly is integrated into Tweetie. If it is, I’ve yet to figure out where and how.

Postscript: Marty Shaw twittered me instructions on how to send links to Tweetie using the iPhone bookmark feature — awesome tip.

URL Size: Domain, Path & Total Characters Used

Another issue for many is just how small you can make a URL. A first step in keeping URL size down is if the domain used by the URL shortening service itself is small. For example, popular service TinyURL has the domain of tinyurl.com, 11 characters. But service is.gd comes in at only 4 characters. TinyURL will always be at a disadvantage over is.gd because there’s simply no way for it to reduce its domain name.

Of course, after the domain name comes the remaining part of the URL, the “path” portion or what on a normal web site would be the page name. The path size for services typically ranges from 3 to 6 characters. If you really want to stay small on character counts, go with a service that uses a short 3 character path.

Keep in mind that over time, even services offering short paths won’t be able maintain this. That’s because there are only so many 3 character combinations that can be used (is.gd has a nice tutorial about this). When those have all been used, a shortening service either has to recycle old combinations or expand the number of characters used.

Recycle URLs?!!! Could that happen? Potentially, yes — a service could reissue a URL that’s been used before. They don’t seem to do this, however. Unfortunately, only one of the sites I visited explicitly said that recycling was not done — is.gd. While the others almost certainly don’t recycle, they should be clear about this.

Overall, if it’s essential to save every character, go with a service that currently uses fewer characters for the path. Otherwise, seek out a service that keeps its domain as short as possible. But as I’ll get into, a short domain doesn’t take away from other issues you may wish to consider.

On the chart, domains that are 15 characters or shorter are shown in green for good; red means they have 16 or more. Characters include the http:// prefix and the trailing / before the path. Note that with Snurl, it produces a longer snurl.com URL from within Twhirl but on the site itself, it uses a shorter sn.im domain. I’ve gone with the shorter figures for the chart.

Further along the chart, you can also see the current character counts for paths generated at each service — and next to that, total characters for short URLs. If you’re after the very smallest URL possible, look at the total characters column. However, I’ve not given any “good” or “bad” credit for either of these columns because they will change (I went with orange to indicate a neutral opinion). Any service offering short 3 or 4 character paths now will, if it becomes popular, grow to using 5 or 6 character paths.

If you’re thinking long-term, especially with tracking in mind, then looking at the total count is the wrong figure, as that will change over time. Instead, the domain character count is more important.

Custom Or Vanity URLs

Typically, URL shorteners make up a random path for your URLs — those /6Q8 or /qu03e3 and other boring sounding names you’ve seen. Some of them allow you to override this, so that your path can have a “custom” or “vanity” ending. That means you can have a custom name for your URL, such as these that lead to Search Engine Land:

• http://bit.ly/seland
• http://cli.gs/seland
• http://kl.am/seland
• http://sn.im/seland
• http://tr.im/seland
• http://budurl.com/seland

I think it’s a nice feature for a service to allow custom naming, so those that provide this are marked green on the chart — those that don’t, red. And here’s a tip. Once a particular custom name is taken, it’s gone. If you want to protect your brand, you might register any vanity URLs you’d really like to have.

Sharing

Some URL shorteners share the most popular or interesting URLs that are being clicked on through their services. Potentially, this is a benefit to using a particular service. It could bring you added traffic. However, in my survey, few of the sites offering sharing did it in a way (or seemed to have enough traffic) to make this a compelling feature choice. It’s a minor benefit, not a huge one, at the moment. That’s why regardless if it’s offered, all services are colored orange.

A quick recap:

• bit.ly shows popular links on its twitter feed here
• kl.am shows most popular links on its home page
• Plurl shows most recent URLs on its home page
• POPrl shows most popular links on its home page
• Snurl shows “interesting snips” on its Browse Snips tab
• Tweetburner shows most popular URLs in last hour and ones that have gotten the most clicks in the last hour
• Short.ie shows most popular and active users
• Eweri is supposed to show popular URLs and latest ones on its home page, but this hasn’t worked for two days running, so I didn’t credit it on the chart.

Country Of Origin

Country of origin (IE, where the domain name is registered, not where the service is based) is something to consider, when it comes to URL shortening services. In order for some of these services to get such short domain names, they’ve registered names in countries off the usual .com beaten track. Armenia, Libya and South Georgia Island are just some of the places used. To date, I don’t know of any country that has a particularly bad track record when it comes to domain names. So this is likely a minor worry. I’ve indicated countries on the chart but marked them all with orange, for neutral.

Handling Parameters

Ever want to retweet a search on Google? Some URL shorteners can’t handle parameters in a URL, all that stuff with ? and & symbols. Consider this search for movie times:

http://www.google.com/search?hl=en&q=movie+times&btnG=Search

Put that into some URL shorteners, and they break — they’ll ignore everything from the ? symbol onward and just point people at the Google home page. So when selecting a service, this is another factor to consider.

In my testing, only Short.ie came up as unable to handle parameters.

What We Use

In deciding which URL shorteners to review, I focused on those that were built into the clients. As I’ve explained, this suggests they’ll have some stability.

There are also some metrics on which are most used such as from Tweetmeme (ff.im is listed there, but that’s simply FriendFeed’s internal shortener and not something that I can see you can manually use. Heavy use is another sign of stability, especially if people are actively turning to a service that doesn’t have the natural advantage of being built into a client.

I also ran a short survey this week, asking people on Twitter what they use. I received 233 responses, which might help people when making decisions among the choices summarized at the end of this article.

TinyURL was most used by 31.1% of those answering (people could choose more than one service). After that, bit.ly was at 25.2%, then is.gd at 9.9%, then cli.gs at 8.7% and tr.im at 5.3% — all others were less than 5%.

I also asked a basic question of why people use the services. Responses were:

Half (49%) said it was to save space. Only 16% said they wanted tracking, though 25% said they wanted both. Among reasons for “other” were to preserve branding or that it was easy (as these are built in).

I also let people leave open comments about why they use these services, what they’d like to see from them or any other thoughts. I’ve grouped them below by common themes:

Comments About Particular Services

• I use bit.ly by choice when I tip to techmeme so i can track. Also really like bit.ly analytics and the fact it tracks twitter/friendfeed mentions/clicks. tinyurl is twitter default – not sure why. twitter should auction url shortners and get some revenue – that will get rid of tinyurl. It is 11 chars long vs bitly’s 6. ff.im is the only choice to shorten when posting from FriendFeed to twitter. Wish they offered bit.ly so I could track clicks. Also dont like URLs that frame content and put ads.
• The only problem with bit.ly is that they often use the lower case L and the uppercase I which look very similar.
• I like the toolbar button that posts to twitter automatically in adjix. Don’t like the ad part though… need to find a different one that has the good feature without the bad.
• Is.gd has the shortest URLs I’ve seen, but sometimes name of the linked site gets garbled (which could be misleading to readers) when I use it on PowerTwitter. The redirects always seem to work though.
• Kl.am is great but I wish I had more control over deleting my urls when I don’t need them anymore. Seems like it would be an easy thing to add in there.
• I like the custom URL and tracking features for tr.im. Hear good things about bit.ly as well.
• Tinyurl is the only one I remember when I want to shorten a link for twitter.
• Tr.im works for me because it has tracking, twitter integration and a firefox plugin. it also just works, haven’t had any issues with it. Tweetie’s bitly integration in the iphone app is great!
• Twitering from tr.im is useful as it’s one less copy-and-paste step. I wish it would give the last couple of tweets and then you could easily make a traceable reply.
• While I haven’t given this decision much thought, I appreciate TinyArrows’ bizarre characters and tiny, tiny urls.

Concerns Over Malicious Links

• You have to trust the person because you can’t trust the link.
• I have concerns about malicious links being hidden in shortened links – got bit by a joke link this week. Not a big deal in this case but could it be a prob?
• Frustrating that you have to click through usually to figure out what the link is going to. Can’t get a clue from the domain name.
• I know it is unavoidable, but I do not like not being able to tell what website I am about to visit or, in addition, not seeing the link for memory purposes. I know they are needed for twitter because of space issues, but do not like them when space is not an issue. IE when people post them in blogs or general articles etc. In those cases, use the full URL. I remember sites by seeing the URL’s and when I cannot see them and all is there is the shortened URL, the link becomes meaningless to me. IE I can never remember where I read or saw something if I have not clicked on a link with the site name in it. Maybe just me, but feel they should only be used for space considerations.
• I love them for saving space, but not knowing where they will redirect me is annoying at times, when I find the time I’ll write a firefox extension or greasemonkey script to change the title attribute to contain the real url by decoding or resolving the shortened urls, like @mattcutts was on about earlier today
• It would be great if the page content was somehow reflected in the shortened URL so that people not used to seeing shortened URLs have confidence they are going to a relevant/safe site. No idea how this would work!
• The problem w/ shorteners is the reader doesn’t know where the link will go to (e.g. YouTube, Slideshare, company site, etc.).

Concerns Over Death Of Links If Service Dies

• A lot of links could die when these services die.
• I’m just afraid one of them is going to disappear, and break all my old links … but tinyurl is too long to use with Twitter, IMO.
• I’m worried the services will die and leave my shortened URLs as broken orphans. If I’m going to switch from the market leader, I want some kind of long-term commitment.
• I’ve discovered that I like using my own short url service for important links – like affiliate links on my own sites – ones that I control. If a shortening service goes down, I don’t want my important links to be at the mercy of that site, but I do want the ability to track the links and get stats. So, I’m using my own service for those important links. For everything else – just sharing of news and interesting links – I use cli.gs. It rocks.
• They’re great. Had no idea there were so many of them. Afraid of the day they will start requiring a fee. Also wonder if a URL you create with it ever expires or if it’s a permanent thing.
• What if they (the URL shortening service) go out of business? Jill Whalen brought up the good point of how an out of business service kills all of that archived traffic potential
• Will the Internet fall over if TinyURL implodes?

Wishlists & Feature Requests

• As a bonus, i especially like to capacity to deactivate/delete links.
• I don’t know why I don’t get google-analytics like tracking capability with bit.ly. Also wish I could use my own custom domain name on top of one of these. Finally I wish they would correlate with search.twitter.com re-tweets (RT) to see if one of my links is going viral.
• I wish there was a url shortener gadget for igoogle that could give you a short url without leaving the page (opening another tab). I prefer bit.ly to tinyurl.
• I’d love for del.icio.us to create a shortcut url for anything I (or anyone else) bookmarks so I could combine social bookmarking and sharing.
• Integration with Google Analytics, pass correct referrers and pass link juice
• I’m looking for good analytics, including city – level location tracking. I would also love for it to be mobile internet compatible (not iphon), so I can check the stats while I’m out. And 301 redirects is a must.
• I’m wondering about integrating them with Google Analytics. Have some ideas, not sure if already being done…
• Integration w/in web browsers.
• There’s so many to choose from, with no real differentiation so I use what’s handy. I’m waiting for something with better tracking to measure with the URL shortening, and using the Google URL builder (or other tracking code) if I really want to track something.
• They are voodoo. No longevity, fragmentation of brand, cannot trust what is behind a shortened URI.
• We need wordpress plugins!!!!!
• Wish there was a search box tool for firefox so I could just throw the long url in there and get it generated
• Wish Twitter offered the service automatically.
• Would like to have tracking from within twitter or tweetdeck

Some Other Assorted Comments

• “Had an interesting experience the other day where my AV [antivirus program] blocked a bit.ly shortened URL (cant remember what it was linking to but It was a URL linked in a tweet from someone Im following – could have been you!). This was the first time that this has happened, although I dont run my AV all the time (shameful, I know) so not seeing it before might have been a timing thing. If this is a widespread pattern I can see this as a pretty big roadblock for the adoption of twitter by the large population of only semi tech oriented internet users that are likely to become the second wave of twitter-ers. It is highly possible that they will not being willing to whitelist a url-shortening service through their firewalls.
• I use url shortening services for space saving, and tracking. I’m not going to tweet or try to IM urls that are 87597893 characters long. I was using tinyurl always until I tried bit.ly yesterday and realized how sexy its analytics were! I haven’t tried the digg bar yet, but I’m curious if they have built in analytics. I think any of these services that continue to innovate and provide functionality besides just a short url will become rather valuable.
• I will start using one with stats as soon as I decide on which service. I’m seeing where short urls with stats are becoming necessary.
• I’ll join the list of people saying “should pass link equity”. also i like to customize the URLs when posting links to my own content, but rarely do so for any reasons other than readability.
• Maybe I am old school but they seem somewhat unofficial. www.cnn.com/top-news-of-the-day has the banding a short url will never have
• Space savings very important, especially when including more than one URL and/or tags in a tweet. Don’t like the ones that frame the resulting page. Haven’t used a tracking URL shortner since zi.ma went under. Would like to find another one to use occasionally.
• Tracking analytics is a must for me, as is a 301. I really like the real-time GUI of bit.ly’s service.
• “Stats are nice but, Do we really need 50 companies developing what is basically a 301 redirect?”
• Terribly useful in a 140 character world and for tracking, and ultimately think that major services will want to offer URL shortening directly
• The minute I tweet I know the reach of my “ad.” Once I see how many clicks I get I can calculate a basic CTR.
• Thought about making my own for personal use…
• Working on making one for my own site to create branding.. why have tinyurl displayed when it could be youfoundjake.com displayed..yeah, i miss some extra characters, but worth it branding wise..
• They’ve made domain resellers and registrars very happy.

Top Choices

You made it! After reviewing the various features, here’s my rundown on recommended services:

bit.ly: It offers all the key features you’d want in a service, with nice stats that show number of clicks over time, what sites are referring traffic to your twitted URL, locations clicks are coming from and Twitter conversations using the URL. It’s also the default in three of the four major clients — or three of the major clients, if you exclude Twitter itself.

[NOTE: AUGUST 2009, BIT.LY IS THE DEFAULT NOW IN TWITTER, ANOTHER STRONG REASON TO CONSIDER IT].

tr.im: Offers all that bit.ly does other than not being built into Twirl. Tweetdeck support is coming soon. Stats are nice. Bit.ly perhaps offers more drilldown on the referrer front, from my fast review. Remember, a more in-depth review of tracking stats will come in the future.

[NOTE: AUGUST 2009, TR.IM ANNOUNCED IT WAS CLOSING, POSTING THIS NOTE ON ITS WEB SITE AS OF AUG. 9

tr.im is now in the process of discontinuing service, effective immediately.

Statistics can no longer be considered reliable, or reliably available going forward. However, all tr.im links will continue to redirect, and will do so until at least December 31, 2009. Your tweets with tr.im URLs in them will not be affected.

We regret that it came to this, but all of our efforts to avoid it failed. No business we approached wanted to purchase tr.im for even a minor amount.

There is no way for us to monetize URL shortening -- users won't pay for it -- and we just can't justify further development since Twitter has all but annointed bit.ly the market winner. There is simply no point for us to continue operating tr.im, and pay for its upkeep.

We apologize for the disruption and inconvenience this may cause you.

LATER, TR.IM SAID IT WOULD CONTINUE OPERATING AND WOULD BE COMMUNITY-OWNED. PERHAPS IT WILL THRIVE GOING FORWARD, BUT THE INITIAL AND DRAMATIC ANNOUNCEMENT OF CLOSURE GIVES ME PAUSE ABOUT ANYONE USING THE SERVICE, IF THEY'RE CONCERNED ABOUT LONG-TERM STABILITY]

[NOTE: Cli.gs announced in October that it would be closing. See also discussion on Techmeme].

cli.gs: Offers the same as the others above, though it is not built into either of the major clients TweetDeck or Twhirl. You do get TweetFeed support. Stats include showing which search engines have spidered your links. There’s a nice “social media mentions” section that show clicks coming off My Yahoo, FriendFeed, Google Blog Search and others.

More Choices

Snipurl / Snurl / Snipr / Sn.im: You kind of want them to pick a name and settle on it (like Sn.im). The service is currently in three clients, but it’s to be dropped from TweetDeck. Within Twhirl, it also uses the long Snurl.com domain. If you want sn.im URLs, you have to go outside the client to make them. Stats are simply the number of clicks — it’s not in the same class as services above. Finally, ZoneAlarm flagged it as spyware. It’s not, but that’s something to consider in case your visitors are running that software and trying to click on your links.

budURL: Being dropped from TweetDeck. Stats look extensive, if you want to pay. One of the longest domain names of any service.

Short.ie: Originally I had this down as not recommended, as it issued a 302 redirect. They got in touch with me and said not so! I tested again, and they are issuing 301s. They also provide custom URLs –I missed that. Both have been updated. Finally, they say that Twitterfeed support is coming. That with basic stats make them a nice alternative choice. However, they still can’t handle URLs with parameters (I tested that again, and it’s still an issue).

kl.am: Not built into any clients and the stats are fairly rudimentary.

POPrl: Semi-basic stats, being dropped from TweetDeck. No custom URLs.

idek.net: Very basic stats only, being dropped from TweetDeck. No custom URLs.

Choices Without Tracking

Don’t care about tracking? Then either of these are options to consider:

is.gd: Built into both major software clients.

TinyURL: The default choice for Twitter, built into TweetDeck and Twitterfeed. But the longest domain of any of the popular services. ZoneAlarm also flagged it as spyware — as with Snipurl, this could cause issues for your visitors.

Not Recommended

The services below don’t do 301 redirection. That’s a recommendation killer, to me:

• Hex.io
• Eweri
• Plurl
• Tweetburner

Avoid These Services

lin.cr: Originally, I had lin.cr on my list, because it was included with TweetDeck. But I was shocked when I tested it — so much so that I won’t even link to the service.

Rather than doing a 301 or 302 redirect, lin.cr spits out a 200. That’s server code for “OK,” or file has been found. In other words, lin.cr is telling browsers (and search engines) that your URL is actually located on their server.

But how does it redirect? It uses a meta refresh tag set to 0, which might be interpreted as either a 301 or 302 redirect, depending on the search engine. That forwards humans to a new page — but by loading up its own page, it also generates a pop-under paid ad using JavaScript. Meanwhile, search engines that can’t or don’t execute JavaScript get shown a different page, one that appears to have a paid link on it.

DiggBar: Just launched this week by social news site Digg, DiggBar seems at first glace to be a URL shortener. It does shorten URLs. You don’t even need to visit Digg to use it. You just enter a URL after the Digg.com domain to shorten it. For example, do this:

http://digg.com/http://searchengineland.com/are-our-brains-becoming-googlized-15421.php

And you get this:

http://digg.com/d1e5BK

Sounds cool, especially because it puts tools for those who are really into Digg at the top of your page, such as the ability to read comments from Digg about your page (if it has been submitted there) or the ability to Digg the page you’re on.

The fun stops there. Like lin.cr, it does a 200 code. That means the page is actually on Digg itself — they’re making a page with the DiggBar and pulling in your content without permission into a frame. That’s not illegal, but it’s a tactic that died off years ago. It also means that if you use the Digg short URLs, none of the link credit passes to your page. It’s all kept with Digg.

There’s no need for you to give Digg all your link credit. If you want to shorten your URLs, use a service that does a 301 redirect.

By the way, if you clicked on the short DiggBar example above, you would have seen the DiggBar load on top of a Search Engine Land page briefly, then disappear. This is because we use a frame busting script that says:

if (top.location!= self.location) { top.location = self.location.href }

(I can’t get the full code to display so search for this in the source code of this page, then copy from the opening JavaScript tag to the closing one, if you want to use it)

The code clears any site trying to frame our content. To see how a page looks via the DiggBar when it isn’t cleared out, try this link:

http://digg.com/d1nYVs

That puts the DiggBar on top of Google.

Postscript: See The Growth Of Framebars & Kevin Rose On The DiggBar for more about the DiggBar, framebars growing in usage and busting techniques.

Open Letter To Twitter & Twitter Clients

I think Twitter and those who build Twitter clients have some responsibility in the URL shortener choices they offer. In particular, I think they should review carefully what they build into their clients and also allow users to add their own custom choices, if they so choice. In particular:

Twitter: TinyURL is ironically the longest domain name of all the URL shorteners. Let Twitter users have an account option to use another shortener, if they so choose. It would also be nice if URLs could be twitted without counting against the Twitter character count at all. Then we wouldn’t have to worry about all this URL shortener madness.

Twhirl: I’m hugely disappointed that the client I use and love added the non-shortener of Digg within a day of it being released plus made it the default choice. Was there no review of what the DiggBar actually does? And why make it the default over better choices?

TweetDeck: Similar to Twhirl, was there no review of something like lin.cr being made a choice for TweetDeck users, the vast majority of whom would not assume it was generating pop-under ads? It’s being dropped in the near future. Let’s hope those that remain are regularly checked.

Further Resources

In working on this article, I came across an number of other articles that you might wish to check out:

• URL Shortening: Hashes In Practice – From Coding Horror, it explains how the shortening is done
• URL Redirection – How it works, according to Wikipedia
• 90+ URL Shortening Services – From Mashable in 2008, covers even more services than above
• URL shorteners reviewed – From Malcolm Coles last month, services are charted showing if they provide tracking, custom naming and URL length.
• 11 Best URL Shortening Services – Vote Your Favorite – From last month at the Online Marketing Blog, services are listed in a nice chart summarizing features such as custom URLs, 301 redirection, tracking and other features.
• on url shorteners – From Delicious founder Joshua Schachter, covers a variety of issues with URL shorteners and why they might be bad for the web, with some suggestions for improvements. Also see discussion on Techmeme. Again, if Twitter were to simply not count URLs against a tweet’s character count — or provide its own shortener — some of these concerns would go away.

I asked Matt if this means we should go back to our existing sites and change URLs without keywords to have keywords in them. Matt told me, “If you’ve got an existing solution that works for you, it’s not really worth going back to change your urls. It may worth considering when you’re doing a new site.”

I then asked Matt if all new sites should make sure to have keywords in the URL. Matt answered, “It makes sense if a) it’s easy for you to do in your content management system, and b) the keywords are useful and descriptive–definitely don’t overdo it.”

Here is the video, it is worth watching:

The question essentially asks if Google views the redirecting of many domains toward one domain as a sign of suspicious activity. Matt’s answer is that there are many companies — Google included — that own a lot of domains and have valid reasons to point them at the main domain. The additional domains might be typos or misspellings, or extra domains that are related to a company’s brand.

But he also says that 301s can be abused as a spam tactic, and that having a “ton” of domains redirecting to one domain might cause Google to take a look and see what’s going on.

Here’s the full, 90-second video below.

I don’t want to stray too far off into a tangent, but terminology has been rapidly shifting about, so I’ll touch on that briefly. “Domaining” is the practice of buying domain names, with the intent of later selling them at a profit. “Domaineering” is a relatively new term the industry is employing to refer to obtaining domain names to use as an advertising medium.

Frequently, domaineers seek to buy valuable keyword domains that people might reasonably go to directly as “type-in traffic” and then park ad content on them, making PPC revenue off the clicks of users who come to the site.

Cybersquatters are unethical domainers or domaineers who obtain trademarked and service-marked terms as domain names in order to make money off of the intellectual property of others. In most cases these days, cybersquatters cannot compel companies to buy back their own marks as domains, since those companies can often force the cybersquatter to relinquish the domain without profit. So, cybersquatters are more frequently taking the domaineering route, buying up domains, and keeping a low profile while profiting off of the clickthroughs of the ads targeted to the victimized brand names. (Such variant brand name domains can also be sold for profit on the gray market.)

So, how do they get away with it—why aren’t they noticed?

Unethical domaineers are highly adept at generating variations upon brand name domains. They register common misspellings of brand name domains, frequently occurring typos, lookalike domains and related stems (registering a plural for a singular term, for instance).

When I used to work for Verizon (now Idearc) Superpages.com, I occasionally assisted the intellectual property department in policing and referring infringing domains over to them for handling. Although back when the new company name was launched Verizon proactively registered a great many of these variant names and used MarkMonitor to help watch for more, there were always new product names being introduced that allowed openings for cybersquatters and character combinations that their monitoring missed. (Not to mention, the IP department seemed more hyper-focused on “Verizon” name derivatives, and less on satellite brands like “Superpages.com”.)

Over time, I’ve run across a great many cases of cybersquatting—not just with Verizon, but also with a really large number of major name-brand sites.

It took me only a few minutes to find some examples like these:

• www.cokacola.com
• www.veriz0n.com
• www.verizpn.com
• www.g00g1e.com
• www.1bm.com
• www.micr0s0ft.com
• www.go0gle.com
• www.g0ogle.com
• www.googlw.com
• www.microosft.com

While you can imagine that these major name brands like Coca-Cola, Verizon, Google, IBM and Microsoft are all big targets for this sort of thing due to their high popularity, lesser brands are also targets for this and are frequently far less well policed. (As of the time of writing this article, none of these domains are pointing to the official sites of the brand names the domains are based upon.)

Many of these types of domains are essentially stealing brand traffic—they are brand parasites. Even worse, a number of the major brand companies involved are actually paying the parasites to do this! You can often find PPC ads from the victimized companies appearing on the typosquatting sites, and money also passes to innocent affiliates and distributors whose ads also appear on these sites. Innocent or no, the money often should be going to you directly rather than going through them due to a bogus domain.

Stop paying parasitic middlemen, and take back your brand! Force the bad guys to relinquish infringing domains, and get them pointed directly at your site!

Using their same methodologies, seriously consider registering other variant names and 301 redirect all of them to your main site.

While an individual misspelling may bring you relatively small amounts of traffic, this sort of long-tail-brand-traffic can definitely add up over time. A single brand name can have quite a lot of potential misspelling and typo variations as well—in bulk, the traffic from all of these could actually give your site a small bump up.

Also, if you proactively block the unethical domainers from nabbing your brand-variation domains, you’ll save money you’d pay to them in advertising fees and money you’d pay to affiliates, and your legal department would be saved considerable time and money in the long run.

Just as a best practice, misspellings of your brand/domain names should be registered by you and 301 redirected to your main domain.

Here’s a list of types of domain name variations that squatters exploit:

• Word stems – plural and singular forms of words, other versions associated with the word (example: ”macy.com“ );
• Misspellings – (common/uncommon misspellings, phonetic spellings)
• Typos
• Versions of words with various letters dropped off
• Versions of words with extra letters inserted (such as adding extra “www” at beginning of names) Ex: wwwgoogle.com
• Transposed letters
• Lookalikes: domains with various letters exchanged for other characters which closely resemble them
• Other permutations – multiple-word domains with dashes or underscores separating the words
• Homophones or phonetic spellings (example: EyePhone.com)
• Domains with another Top Level Domain (“TLD”) suffix, such as .NET, .BIZ, .INFO, .FR, .BE, .IT, .DE, etc.
• Domains of your name translated into the equivalent word(s) in another language?

Since there can be various combinations of all of the above, there are many combinations possible for a company’s brand names! And, don’t just limit it to the company name alone. All a company’s marks should be checked for this sort of thing.

Nearly everyone who owns a domain name has got a domainer cleverly receiving traffic when people mistype the “.COM” part of their domain names—if you leave out the “O”, you end up getting redirected to an affiliate site, an ad, or to a parked domain page. Ex:

• www.microsoft.cm

This is because the famous domainer, Kevin Ham , made a deal with the country of Cameroon to allow him to wildcard any unregistered domain name traffic for the .CM TLD.

And, there have long been rumors he might eventually persuade the Colombian governing body to allow a similar arrangement with the “.CO” TLD!

There are tools out there which can help you to automatically generate lists of variation words to use as domain names, if you want to police your brand.

One of the best tools for this used to be a Microsoft research project, Strider URL Tracer with Typo-Patrol—it was once my favorite tool. Unfortunately, this project is no longer being supported and upgraded by Microsoft.  I wish they’d bring it back!

There are others out there as well such as this one. There’s also a number of desktop software packages which do the same sorts of things.

If you’re not adept at this, though, you might also consider hiring someone who can assess your unique situation, generate the brand name permutation lists, check them for squatters, and then recommend ongoing strategies to you. As you can see, these guys are clever, and not everyone can think like them.

If you have a major name-brand and are a publicly traded company, it really behooves you to find the cases of cybersquatting associated with your brand names, and force the owners to relinquish them to you (unless it is a “fair use”). This is a necessary part of protecting your marks. Buy up other variations of your domain as well, and 301 redirect all of these back to your main homepage. By doing this you can save money by cutting out the parasites and middlemen, and you can also increase your organic traffic through all those long-tail-brand domain referrals that start rolling up into your homepage.