The Rise (And Fall?) Of Real-Time Search

Blogging and micro-blogging represent a stream of real-time web activity; from important real-life events to random personal thoughts, more Internet users are putting what they see, hear, think, and feel into this stream. Twitter, in particular, is a river of information. Breaking news is often reported there first, ahead of any other media — online or not. Just last week, Danny Sullivan wondered why the major search engines don’t offer a search service that taps specifically into this stream of information.

Two other services are aiming to do just that, but Twitter itself could prevent real-time search on other platforms from having a chance of success.

Twingly offers what it believes is the first federated microblog search service: Twingly Microblog Search. The search engine includes Twitter, Jaiku (which Google is officially abandoning), Pownce (already shut down, but Twingly has six months of archives),, and several smaller microblog services from around the world.

Real-time search across multiple platforms sounds promising, but because Twitter is the 800-lb. gorilla in this space, a Twingly search will tend to look a lot like Twitter Search.

Twingly and Twitter comparison

That’s not Twingly’s fault; it’s a reflection of Twitter’s dominance.

Meanwhile, does the name PubSub ring a bell? It launched years ago as a real-time RSS/Usenet/breaking news search tool, then shut down in 2007. ReadWriteWeb reports that PubSub is coming back and hopes to be a real-time blog search service.

ReadWriteWeb says PubSub, which is in a closed beta right now, should get updates “only seconds after a story” is published because it’s part of the Ping-o-Matic service. That puts it in competition with Google Blog Search, My Yahoo, Technorati, and other sites that get pinged when new blog posts go live. PubSub, the RWW article says, also accesses some Twitter feeds. But those updates are slower, presumably because PubSub doesn’t tap into the Twitter API.

The Twitter API allows external services to pull in what it calls a “Firehose” of activity on Twitter — all the posts as they happen, and more. Without using the API, it might be difficult, maybe impossible, for search engines to gather Twitter activity in a timely manner.

And there’s the fly in this soup: For all the effort to put value into real-time search — particularly where Twitter is concerned — Twitter itself may be putting an end to those efforts just as they’re getting started.

Jesse Stay, the founder of SocialToo, wrote yesterday that Twitter has announced new limits on its API that will take effect this week. Stay writes that will have a dramatic impact on the ability of other services to tap into the Twitter stream:

“… this is going to hurt every app out there. I’m arguing that 20,000, or any request-rate limit for that matter, limits any app out there from being able to develop on the Twitter platform, and I don’t see why any able-minded entrepreneur would want to build on it if there’s such a rate limit in place.”

If accessing the stream becomes more difficult, or limited, it may be that Twitter’s own search engine will become the only place real-time search has a chance to grow.

Related Topics: Channel: Strategy | Features: Analysis | Search Engines: Blog Search Engines | Search Engines: Word Of Mouth & Buzz Search Engines


About The Author: is Editor-In-Chief of Search Engine Land. His news career includes time spent in TV, radio, and print journalism. His web career continues to include a small number of SEO and social media consulting clients, as well as regular speaking engagements at marketing events around the U.S. He recently launched a site dedicated to Google Glass called Glass Almanac and also blogs at Small Business Search Marketing. Matt can be found on Twitter at @MattMcGee and/or on Google Plus. You can read Matt's disclosures on his personal blog.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • chiropractic

    As coincidence would have it I was leaving the office when we had an earthquake tonight. Jumped in twitter 1st and then posted info to my site with links to gov earthquake data. Ironically, it was the only result for the quake appearing in google news, until about 45 min after event when NBC Los Angeles info appeared. By that time, hundreds were all over the conversation on twitter.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide