• http://www.daniellegauthier.com Danielle Gauthier

    Interesting – I was just about to undertake the same exercise and then I found the post here. Saves me the work and you were so much more thorough than I would have been.

    At least Google got it right when it pointed me here. Though many celebrity obsessed individuals might disagree with me there.

    I think one of the most important things Google needs to work on, which you address here, is weeding out duplicate content (or even just mildly duplicate content). I think the value of real-time results is diminished when you have to search through multiple articles just to find something new that has not been repeated in a previous result. Too, if there is a sudden spike in articles about such and such term, then clearly no one is searching about a news item from 2 months ago. There should be more intuitiveness with regards to what is presented – clearly everybody today is searching for news Brittany Murphy’s death, and no one cares about the asthma attack her husband had a month ago.

  • Shyam Kapur

    This is an excellent analysis. To get the most value out of tweets and other real-time messages, a more sophisticated technology-based solution like TipTop’s at http://FeelTipTop.com is essential. Other methods will always fall short.

  • http://www.mattcutts.com/blog/ Matt Cutts

    Thanks for writing this article, Danny. While Brittany’s death is a tragic event, I think it’s fair to look at breaking news as a way to measure the quality of real-time search. I can add some useful data on this from Google’s side:

    – The AP reported that TMZ was the first to report the new of Murphy’s death.
    – The article that broke the story appears to be http://www.tmz.com/2009/12/20/brittany-murphy-dies-cardiac-arrest/ and it looks like it appeared at 1:37 PM Eastern.
    – From the data we have, it looks like the first tweet about Brittany’s death happened at 1:40 PM Eastern time.
    – Google started triggering realtime search about 2 and a half minutes later.

    That’s really quite responsive, and it was entire algorithmic. As Danny noted, not a lot of spam intruded as he was watching.

    As John Markoff has noted at http://twitter.com/markoff/status/6225910180 one of the main strengths of Twitter is to flag a big event. But certainly noise does happen after that (both because there’s only so many known facts, plus people start to tweet their opinions). I think Danny makes fair points about better tools being needed to search the real-time stream and to highlight the important links/stories. At the same time, the real-time stream worked as intended to highlight a breaking story and to show the flavor of how people are reacting to the event. The rest of the search results are also there to help give important news and context. And even the Google real-time results did a fair job of highlighting news articles, not just tweets.

    Can Google do better? Sure, and I think we’ll continue to iterate and improve. But to trigger real-time results in a very small number of minutes after the news breaks, and with very little spam–that’s a pretty good result for a first-time test of real-time search. I’m glad that the auto-suggestions were fresh enough to help users too.

  • Stupidscript

    Interesting, Danny.

    It would have been good to have assigned another person to do the same with Bing, simultaneously (if you had anyone to assign). Apples to apples, and all. As it stands, we only have your comment about how Bing did no better with their “Twitter Search” feature. It would have been interesting to be able to compare results based on the Twitter API Microsoft is tapping into with results from the more limited API that Google has available, and to see how the two companies compare, seeing as how Microsoft has access not only to public individual pages like Google does, but also to the ‘fan’ and other higher level pages that Google cannot access in real time.

    Personally, I think your report was pretty good, and that Google’s (Matt’s) response was very reasonable. Your reporting on this topic will improve the more you do it (it feels like you were scrambling to define your methodology) and I’m certain that Google’s programming will also improve the more they do it. Probably, Bing’s will, too.

  • bob222