Where Have All The Old Tweets Gone?

Looking for old tweets? You won’t find any that are more than a week or so old on Twitter. Put the blame on all the massive number of new tweets coming in. The engines, they canna hold captain! Or that is, the search index behind Twitter Search can’t hold it all.

NOTE: Since this was written, Google has launched an archive of old tweets. All The Old Tweets Are Found: Google Launches Twitter Archive Search has more about that service.

To see this in action, consider a search for “happy new year,” date restricted to New Year’s Day 2010. You’d expect a huge number of matches on that day. You’ll find none.

In fact, in testing the date restriction command on Twitter Search’s advanced search page, I found that it doesn’t seem to be functional. I looked for any tweets containing “happy new year” since Monday and received only four matches. In contrast, if I remove the date restriction, I can see there have been at least 1,500 tweets with those words today alone.

So are the tweets still there in Twitter Search, just inaccessible because of a bad search function? No. I did a different test, a regular search for the word “pedometer.” To find the oldest tweet, I went back page-by-page as far as I could. Actually, I did the search, then changed the “page=” part of the URL to a higher page value. The oldest tweet is six days old. Any that are seven days or older don’t appear.

This matches to what a reader asked about earlier this month. They’d emailed us:

Why am I the only person upset with Twitters cutback on their search engine results? Earlier this month I noticed that they were only displaying the results from previous two weeks. Today I see they have slipped to just showing results from previous 7 days? I can find no mention of this anywhere including their blogs.

Out Of Room In The Search Index

I checked with Twitter and got back this response from Doug Cook, Twitter’s director of search:

In response to your reader’s question, we weren’t growing our search index as fast as the tweet volume was increasing, so it started to represent a decreasing amount of time.  In the last couple of days we increased our index size somewhat, so the amount of time will go back up, but there’s going to be a natural “yo yo” effect as the tweet volume increases in advance of our next index size “jump.” As you might guess, we’re working on making this far, far better.

Increasing indeed. This week Twitter cofounder Ev Williams tweeted that the service has seen its highest usage ever. All those tweets have to go somewhere, and that impacts the search index.

The search index is like a big book that contains all the information that the search engine looks through. With Twitter, when you’re searching for something like “pedometer,” if flips open that book and starts listing all the pages in it that mention that word — those “pages” being individual tweets.

However, an index can run out of room, and that seems to be what’s happened in this case. There wasn’t enough room to keep all the tweets recorded in the index, easily accessible for searching, so only about a week or so seem to be present. That’s why you can’t “page back” beyond a certain date. The problems with date restricted searching seem to be a combination of both the index limitation and probably a bug.

Meanwhile, Mashable noted this week discovering that the number of tweets shown for some individuals appears to have gone down massively, when the figures should have gone up. Cook said that’s almost certainly unrelated to the index issue and a separate problem Twitter is exploring.

Old Tweets Are Safe & Online — Twitter Search Just Can’t Find Them

Don’t worry that your old tweets have gone missing for good. They’re still out there, just not immediately accessible to locate through search or by working back page-by-page.

I bolded the words “through search” to stress another point. The old tweets do exist online, if you know where to find them directly. It’s just that Twitter Search itself can’t point you to them. For example, here’s a tweet I did from July 2009.

Some Search Alternatives For Twitter Archeologists

What if you do need to find an old tweet and don’t know where to look? Some suggestions:

Google offers its own real time search (see Google Launches Real Time Search) as does Bing (see Up Close With Bing’s Twitter Search Engine). Unfortunately, neither of those services allow you to narrow your Twitter search to a specific date range.

“Regular” Google can help. Do a search like this:

google os site:twitter.com

The site:twitter.com portion makes Google show only things on the Twitter site (mostly updates people are making) for the words you specify (google os, in this example). After that, you can use the Show Option feature to choose “Recent Results,” then “Sorted By Date” to get the freshest posts at the top. Here’s an example:

Searching Twitter With Google

The problem is, Google doesn’t always get the datesof a tweet correct, I’ve found. But it’s something to try.

By the way, if you want to search for tweets from a particular person, try it like this:

site:twitter.com/dannysullivan

Just put the person’s Twitter name after the site:twitter.com/ part, as I’ve done for myself above. It works to focus the results to matches from just that person. You can also narrow further to get tweets from that person about a particular topic, such as this:

site:twitter.com/dannysullivan bar sign

Over at Bing, the same techniques I’ve shown also work, though Google tends to be more comprehensive and consistent in what it shows, I’ve found.

At Yahoo, they also work — though if you do site:twitter.com or site:twitter.com/username without following those with any search terms, you’ll be kicked over to the Yahoo Site Explorer area. You still get results, but it’s through a different interface.

The best way I’ve found to dependably locate old tweets has been through FriendFeed. The caveat is that this only works for people who have imported their tweets into FriendFeed. As I do this, FriendFeed works as a great backup service for me to locate my own posts.

Here’s an example of the key commands:

service:twitter.com from:dannysullivan bar sign

In that example, I’m saying find any tweets from Twitter done by the FriendFeed user dannysullivan (which is also my Twitter name) and which has the words bar sign in them. Voila — I get a nice list.

Still, there are issues. Consider this search:

service:twitter.com from:graywolf google

That should get me any tweets from the FriendFeed user named “graywolf.” Instead, I get back matches from Google Reader. And the Graywolf on FriendFeed isn’t the same as the Twitter user Graywolf (Michael Gray). So if that was who I was after — and I was — I’m out of luck.

Related Topics: Channel: Social | Features: General | Top News | Twitter

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Shyam Kapur

    Am I missing something? Perhaps you are running out of things to write. For at least 6 months, it has been impossible to go back and find tweets older than a week through search in Twitter whether you use their web interface or any of the API’s they provide.

  • @glyka

    If you want to keep a record of your tweets and search for an old one in the future, I suggest you use the RSS located on your twitter profile page, this way they can be downloaded to your Google Reader or something. Or you can also do it through friendfeed.

    For searching twitter hashtags or keywords we will have to stick with the google and bing search :(

  • http://www.michael-martinez.com/ Michael Martinez

    I’ve been aware of the limitation for several months as well. I merely assumed the people at Twitter didn’t consider the old tweets to be relevant to current searches. I’m not sure what the value in providing a full index would be to users.

  • http://lisapadilla lisapadilla

    A very informative post, thank you.

    I too was frustrated with this problem for a long time, with a lot of services. Grabbit.net, a company I founded, is about to release a product which allows you to read Facebook, Twitter, RSS, and email — saves all of it — and allows you to search across it all too. For now, this service is free. If you’re interested in kicking the tires on the pre-release, email info at grabbit dot net and I’ll send you an invite.

  • http://www.linkedin.com/in/BarbChamberlain BarbChamberlain

    Just what I needed! I have specific tweets I use in presentations but some of my examples are getting a bit old. Unfortunately I didn’t do a screen capture when I should have, and now can’t page back far enough to get them.

    However, this didn’t work for me in looking for a really old tweet (August 2009) even though I know the account name and exact wording.I hope Google catches up with the old stuff. Frustrating, given that we worry about things living online forever, when I can’t find something I know was there!

    For tweets from the account I manage for work, I use backupmytweets.com in order to create an archive (we’re a public institution & have records retention requirements).

    @BarbChamberlain
    Director of Communications and Public Affairs
    Washington State University Spokane
    @WSUSpokane
    http://www.spokane.wsu.edu

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide