• Winooski

    Hats off to you for this exhaustive article about a very “of the moment” (in every respect) subject.

    The issue now, as it has been since the days of Infoseek’s real-time search, is spam: http://searchengineland.com/twitters-real-time-spam-problem-20614 . Until there’s a robust way to filter out spam, real-time search will end up suffering from the Tragedy of the Commons.

  • http://ninebyblue.com/ Vanessa Fox

    Interesting definition of real-time search. I have always thought of real-time search as the ability to search anything that has just that second been posted (so similar to Google’s “fresh” results only faster and more comprehensive). Your definition combines search technology (the ability to grab stuff quick) with user behavior (the trend towards posting quick, immediate thoughts).

    Which makes sense because the primary reason people want real-time search tends to be that they want to find real-time information (things that are happening right now). So with your definition the “search” in real-time search actually covers the the behavior of the content creator (and the platforms on which they can perform that behavior), the search engine, and the searcher.

  • Kimbal

    Nice post, Danny. I’ve just posted a response on the OneRiot company blog – http://bit.ly/VObyL – had some counterpoints to your arguments :)

  • http://politicoTracker.com pTracker

    I’m afraid I have to disagree with your terminology (and disclose up front the reason is that our iPhone app delivers “real-time” news search results as one of our differentiators).

    Quote:
    “To me, “real time search” should be reserved for searching what’s being tweeted, what people are talking about, what they’re microblogging. “Real Time Chat” or “Real Time Talk,” if you need a different name for it. Maybe “microblog search.” Just as news search covers what’s being published from news sources, real time search for me covers what’s being said in real time, not links that are being passed around.”

    The problem as I see it is that “news search” is too broad, and “real-time search” is too generic. In the phrase “real time search”, the “real time” refers only to the freshness of the search results, not to anything about the origin of the content or how quickly it was published.

    Likewise, “news search” talks only to the origin of the content, and not how old it is. Thus, daily news aggregators fall into this category, as do historical research tools. Our app has access to a steady and ongoing stream of world-wide news, which is processed in real time as it arrives (as opposed to batched) and available to our users almost instantly. So the phrase “real time search” is highly applicable. How long it took to research an article has no bearing on how quickly a user has access to it once it was published. Since this is a feature of our app which we believe users value, we will continue to promote it as yet another of our differentiators from news aggregators. See http://politicoTracker.com for more.

    If you intend to include in your new term the notion that the content being searched is produced in real time, then a new term should be used (this is after all a new concept). How about “reality search”? Or, “concurrency search”? Or, perhaps, “immediacy search”?

    Thanks for listening
    http://twitter.com/_pTracker

  • http://www.aafter.com aafter

    Good posting.
    At AAfter search we provide real-time results depending on the query. Try to ask us a question, and see how it works. However, we are far from what we want to do with real-time search that is useful for our visitors.

  • http://www.gnip.com bpm140

    I’ve never heard the latency of content production brought up in the RTS discussion before, but it makes a certain amount of sense, as it’s analogous to traditional TV news where 20/20 or 60 Minutes news features would never be considered real-time coverage (especially compared to, say, watching a white Ford Bronco get chased at 3p in the afternoon on every channel… but I digress…)

    One of the key questions though is how does that definition of real-time affect the value of the content. Many people would argue that the proliferation of 24-hour news networks has led to a lack of substantive coverage, because people are placing more value of timeliness over content.

    Yes, it takes longer to write a couple of paragraphs, but the result is often far more insightful than what can be encapsulated in 140 characters.

    I hope we’ll continue this conversation tomorrow at the CrunchUp.

  • http://www.brickmarketing.com nickstamoulis

    This is a great analysis of the real time or social search engines on the market, I personally have used Friendfeed and Collecta a good amount, here is a blog post that I wrote regarding real time search and Collecta:
    http://www.searchengineoptimizationjournal.com/2009/07/06/real-time-search/

    Anyway, I personally think Google (and the other major engines) should just integrate the social or real time search results into the SERPs, much like universal results or even add another link at the top to be the following:

    Web Images Videos Maps News Shopping Gmail Social

    I think this would give visitors a much better user experience and would prompt more visitors to use social search.

  • jngross

    What an incredible piece of work putting this all together. Thanks for your insights on these various problems. From my perspective the main problem is signal to noise ratio. Yes, spam is a part of it, but the other huge problem is duplication, and identifying when new content is really “new” when it is added into the system. Simply keeping track of the time of publishing into the web is not determinative, ‘cuase people (and robots) repeat information. Duplication adds no real information, yet it probably takes up half of the web’s content.

    What this means is in the typical sequence of events:

    event1 (MJ ambulance) – observation1 (I see it live) – report1 (through Twitter)

    the far right side of the pipeline starts to clog rather rapidly with people repeating the story; soon the web is clogged with a gazillion stories about the ambulance

    now:

    event2 (MJ dies) – observation (paramedics report) – report2 (through Twitter)

    this comes into the pipeline 10 minutes later, but is buried by all the noise caused by lingering duplication of the prior story; “teasing” it out is the key, IMO, to achieving “real-time” search of the kind you are suggesting…. ’cause if you want real time knowledge, that’s different than real time “search” if you define the latter to be tied to the temporal qualities of the reporting, as opposed to the underlying event.

    thanks again for a great piece,

    JNG

  • http://www.feeltiptop.com Gregory Martin

    Harnessing real time knowledge, sentiment and consciousness found through social search and web search should be a goal of search engines.
    Searching people’s experiences published on Twitter, individually or on the aggregate, viewed as Tips (Pro), Pits (Cons) and Remaining (Fact-oriented) messages is what TipTop search is all about. Check out TipTop’s search results for some popular topics in this article:
    http://feeltiptop.com/dannysullivan/
    http://feeltiptop.com/realtime search/
    http://feeltiptop.com/social search/
    Only TipTop can let you know how people feel about what they are really searching for http://feeltiptop.com/searching for/.

  • http://drburix02 drburix02

    Here’s another brand new real-time search engine that can effectively retrieve information streams (rather than individual posts): Feedmil.com
    Feedmil provides the broadest possible coverage in terms of search results by monitoring all sorts of stream types – microblogs, blogs,podcast, public & social media.
    In addition, users can conveniently set popularity range for search results to
    explore high quality but less popular information streams more quickly.