Yahoo News was the web’s number one choice last week for information about the death of entertainer Michael Jackson. In a post on the Hitwise blog, Heather Dougherty shows a chart detailing how the major online News & Media sites fared:

Late last week I wrote about how each of the major search engines handled the extraordinary search activity that followed the deaths of Michael Jackson and Farrah Fawcett on the same day. Yahoo is in a unique situation as an actual content provider with writers and editors, and some of those staffers were called in from home to help Yahoo cover the breaking news. The chart above suggests the effort was worth it.

But perhaps the big winner on the day was TMZ, the entertainment site that first broke the news of Jackson’s death. Hitwise says TMZ traffic hit a three-year high on Thursday, with 5x more visits than the day before.

An extraordinary day of breaking news on Thursday led to record-breaking traffic spikes as people searched online for information about the deaths of Farrah Fawcett and, especially, Michael Jackson. And just like their counterparts in traditional media, the news divisions of Google, Yahoo, and Bing responded with sometimes extraordinary measures to ensure they were giving searchers the most accurate and current news available.

Below, a look not only at the extreme traffic spikes that took place, but also an insider’s look at what happened as each search engine — and Wikipedia — grappled with the demands of a nearly unprecedented surge of interest in the day’s breaking news.

Google: “An all-hands-on-deck moment”

“Thursday was a pretty out-of-the-ordinary day.” That’s how spokesperson Gabriel Stricker describes the scene at Google’s headquarters while millions of people were online trying to find out what happened to Michael Jackson. Google has written about what it calls an “outpouring of searches” about Jackson. Stricker says Google saw a wide range of queries — like “michael jackson died” and “michael jackson hoax” — that peaked at about 3:00 pm PST.

The rush of traffic was so severe that Google initially thought it was under attack.

“That was an all-hands-on-deck moment,” Stricker says, “until we were able to determine that the original assessment was wrong, that it wasn’t an attack.” The massive spike in searches “tricked” Google News into showing an interstitial error page for about 25 minutes.

Google says the mistaken attack was its only hiccup yesterday, and that they saw no need to manually adjust results so that searchers got the right information. “The spike in traffic is an indication that we accomplished what we set out to do,” Stricker says. “People came to Google looking for an answer to a specific — and in this case, rather sad — question, and they got it quickly.”

Google hasn’t said yet how Thursday’s traffic compares to other important news events. Their blog post does say that Google “saw one of the largest mobile search spikes we’ve ever seen.” And Google Trends labeled Thursday’s searches for “michael jackson died” as “volcanic.”

Yahoo: “This demanded that we take our coverage to the next level”

Thursday was a record-breaking day for Yahoo. Their story, “Michael Jackson rushed to hospital,” received 800,000 clicks in 10 minutes, making it their highest-clicking story ever.

Yahoo also revealed that Yahoo News set an all-time record with 16.4 million visitors, beating the old record of 15.1 million set last election day. The four million visitors between 3-4 pm PDT set an hourly record.

Things were no less busy inside Yahoo headquarters. Richard Vega, Editor of Yahoo News, also described it as an all-hands-on-deck situation, going so far as to bring in staff on their days off. “After we saw initial reports that Michael Jackson had died, we immediately devoted all resources to the story and called staffers at home to help,” Vega says.

As a content destination and a news organization with writers and editorial staff, Yahoo took a more hands-on approach to packaging information for its users — even sending staff out to report live from Los Angeles. Says Vega: “We made sure to include the main stories and sidebars from AP and Time magazine. We had video clips from ABC News. We created slideshows. Since Michael Jackson had died in L.A., we sent out two editors to the UCLA Medical Center to interview and take photos of the fans who were gathering outside the hospital. In addition, one editor sent Twitter updates from the scene. This was a unique moment in history, which demanded that we take our coverage to the next level.”

Yahoo News recorded 175 million page views on Thursday, its fourth-highest total (after Inauguration Day, the day after the Inauguration, and Hurricane Ike). A blog post in Yahoo Music has received more than 21,500 comments as I write this. And Yahoo says Flickr has seen more than 4,000 Michael Jackson-related photo uploads in the past day. One poignant Flickr photo shows Times Square at a standstill as the offline world reacted the same way we did online.

Bing: “We rolled out a ‘news go big’ experience”

Like its competitors, the Bing search team was paying close attention to Thursday’s news as it unfolded, and doing its best to make sure searchers got the information they wanted. Jamil Valliani, a senior program manager for Bing, and Todd Schwartz, group product manager for Bing, said their effort included “the extended search team, including engineering, product management and marketing.”

They say Bing “definitely saw a spike” in traffic on Thursday, but they don’t have any data to share at the moment. “We get more feedback and see more engagement from consumers for bigger news stories, so we do have to spend more time than average reviewing this feedback and taking it into consideration.”

Some of the feedback for Bing’s Jackson-related search results wasn’t good. On Search Engine Journal, for example, Loren Baker pointed out that Bing’s search results led off with Michael Jackson photos, while news links were at the bottom of the search results page:

Valliani and Schwartz describe what happened: “In general, our rule is not to interfere with the normal algorithmic operation and to note any interesting or unexpected behaviors to be addressed in future upgrades of the product. The only exception to this is for major news events where we see unusual volume, and the results are clearly not being ranked in a relevant way. In these cases we can respond more quickly to how we perform the ranking. This was the case yesterday with Michael Jackson in particular, where we quickly rolled out what we call a ‘news go big’ experience to make sure we were providing appropriate coverage for this significant and sad event.”

Search experiences on other sites

The extraordinary online search for information about Thursday’s news wasn’t limited just to the major search engines. The Wikipedia page about Michael Jackson saw an enormous jump in pageviews on Thursday, and even more on Friday according to Grok.se, an unofficial Wikipedia traffic stats site.

Since Wikipedia pages are open to community editing, Wikipedia took unusual steps to deal with the situation as rumors spread Thursday afternoon.

“The Jackson page was temporarily ‘protected’ to prevent any editing as soon as the rumors started,’ according to Wikipedia administrator Jonathan Hochman. “There was a community discussion about how to handle that. The idea was to prevent the article from going back and forth, or being the subject of a hoax, until the story was verified.”

Twitter was a hotbed of Jackson-related searching and conversation. Twitter co-founder Biz Stone told the Los Angeles Times that there were nearly 5,000 Jackson-related tweets per minute on Thursday afternoon. “We saw an instant doubling of tweets per second the moment the story broke. This particular news about the passing of such a global icon is the biggest jump in tweets per second since the U.S. presidential election.”

Likewise, Facebook reported a tripling of the number of status updates in the aftermath of Jackson’s death.

The final word about this extraordinary day belongs to AOL, whose AIM messaging service was knocked offline for 40 minutes Thursday. Their statement begins like this: “Today was a seminal moment in Internet history. We’ve never seen anything like it in terms of scope or depth.”

Postscript: See Google Thinks Michael Jackson Died At Age 65 In 2007 for how Google’s currently listing the “wrong” Michael Jackson in response to searches for “michael jackson died.”

This week’s battle royale has pitted newspapers and traditional media against Google and online news. If you somehow missed it, the CliffsNotes version is that traditional media (Associated Press, newspapers, etc.) are accusing Internet sites (search engines, news aggregators, etc.) of diverting traffic that should be going to news web sites, profiting off news content in violation of copyright law, and essentially sending newspapers into the financial crisis that many are in today.

Amidst all the claims and accusations, Danny took the AP and newspapers out to the woodshed and pointed out what you’d think should be obvious: Newspapers ought to appreciate search engines for sending them so much traffic.

But how much do they send?

Hitwise takes a stab at answering that question with new data that shows search engine traffic (Google, Yahoo, etc.) and other News & Media sites (including Yahoo News and Google News) are easily the top two traffic sources for news and media web sites.

Hitwise indicates that Google News and Yahoo News are Nos. 2 and 4, respectively, in the list of referring sites inside the News & Media category (blue line above). That, combined with Search Engine traffic at an essentially equal level (orange line above), indicates that newspapers are biting the hands that feed them.

In fact, the Hitwise report makes an interesting point:

“Branded searches for news properties represent a large share of the top search terms driving traffic to the category.”

This flies in the face of the AP/newspaper contention that Google and other search engines and news aggregators (like Yahoo News and Google News) are diverting traffic away from news web sites. Not the case. As Hitwise’s Heather Dougherty writes, “Although several of the online aggregators are at the heart of the content distribution argument, they do successfully send visits to news properties rather than keeping them upon their own websites.”

There’s more discussion on Techmeme.

Google CEO Eric Schmidt addressed the Newspaper Association of America conference in San Diego this morning. It comes against the backdrop of the AP copyright controversy, rising tension between Google and newspapers and publisher frustrations with their own declining revenues. Some have accused Google of devaluing their content.

Schmidt’s talk was generally sunny and upbeat especially about the role of technology in solving problems. The most interesting part of the talk was the Q&A session. Here’s a summary and general paraphrase of the question and answer part of the speech:

Question: In addition to advertising, you spoke about subscriptions and micropayments. Please elaborate on the last two.

Schmidt’s answer: On the Internet we think you’re going to end up with all three. There’s free television, cable television and pay television. That structure looks to us like the structure of all these businesses with the audiences getting smaller with the pay models. The vast majority of people will only want the free model, so you’ll have to deal with advertising.

Question: You’ve said that news organizations are trusted, have credible brands. Is there a way to look at search so that news organizations with credible brands . . . that the algorithm is tweaked not only for the benefit of the publishers but for the users?

Schmidt’s answer: We actually do that in the case of Google News. There’s a relatively fixed set of sources, which are selected based exactly on the sort of trust you’re describing. In Google search we’re careful not to bias it using our own judgment of trust; we’re never sure if we get it right. We use complicated ranking signals . . . and we change them periodically, which drives everybody crazy.

(He spoke also about the trade-off between popularity and trust or credibility in Google’s general search algorithm and said “we’ve not come up with a way to algorithmically handle that in a coherent way.” In this case “that” is a lesser known but credible source that’s getting less traffic than a more popular source.)

Question: Speak frankly about how newspapers have performed digitally over the past 10 or 15 years. And if you became the CEO of an American newspaper company, what would be the top few things you would do in the digital space?

Schmidt’s answer: I was very impressed by how quickly all the newspapers I talked with in the 90s embraced the Web. They quickly repurposed existing print stories on the Web and created reporter blogs. The criticism if I can offer one is that there wasn’t an act after that. And the act after that is a much harder question. How do you keep engagement; how do you keep from being disintermediated into just a set of stories with your brand on them, which has happened to some newspapers?

If I were involved with the digital part of the newspaper, I would first and foremost try to understand what my reader wants. It’s obvious to me that the majority of the circulation should be online rather than printed. There should be 10 times more readers online because there are no distribution costs.

So the question becomes, how do we get to 10 times more readers online and what do they want to see? My own bias is a technology one; I think the sites are slow. They’re actually slower than reading the paper. And that can be addressed.

Question: AP decided to take a more aggressive approach to intellectual property rights. What impact will that have on Google and intellectual property generally. And how do you see intellectual property rights over time, will they continue to erode?

Schmidt’s answer: We have a multi-million dollar deal with the AP not only to distribute their content but to host it. We have a successful deal with AP and hopefully that will continue for many many years . . .

The resolution of the second question is determined by how you interpret Fair Use. From our perspective, there is always a tension around Fair Use. But ultimately it’s a balance in favor of the interests of the consumers . . .

Think in terms of what your reader wants. These [newspapers] are ultimately consumer businesses and if you piss off enough of your readers you won’t have them anymore. If you make them happy, you will grow them quickly.

There are laws that govern intellectual property. It’s important that you respect the law. We try very hard to respect the rights of the copyright holder. But all of these partially thought through legal theories are now being challenged by the ubiquity of the internet.

Question: Google has been at the forefront of conditioning audiences to where the headline and the extract are enough. Google sits in the middle between the content and the audience. How can the media industry in general partner with Google to help support that professional content, when the headline and extract are enough?

Schmidt’s answer: Everyone has an opportunity to opt-out of this using robots.txt. You should understand that. But we think we can build a [personalized] business with you guys with significant advertising resources, where the advertising is targeted toward the content.

Question: One of the issues that’s still a problem for us is . . . there are no defined standards of what an eyeball is. What is truth to you when it comes to external or internal sources of audience counting?

Schmidt’s answer: We look at clicks and how long people stay on a page. Your question shows us how early we are in this industry. It took many many years for the creation of audit circulation bureaus for magazines. A uniform standard will be developed over time.

Comments: Schmidt was generally open and charming but didn’t offer the newspapers any real, concrete solutions. Being more user-centric is good advice but general. The audience was polite and didn’t really challenge him. I’m sure there was a lot of unexpressed frustration in the room. Google is seen, as much by some as Craigslist, as a villain and destroyer of newspapers.

The truth is much more complex. Newspapers have seen some of their business commoditized by news aggregators, of which Google is merely one. They have also failed to create the user experiences and products that might make them more successful online. Yet they are relatively successful online and among the top sites in their respective markets in many instances. However they suffer from the same tension faced by all traditional media: revenues reside in the traditional product but audiences are increasingly online.

There are no easy solutions but setting Google up as a scapegoat, as newspapers have often done with Craigslist, obscures things that they have control over and actions they can take in adapting to a dynamic media marketplace. With their complaining, newspapers strike me a little like Albert Books in the movie Broadcast News. Even though he was smarter, he “lost the girl” (Holly Hunter) to the more attractive but dimmer character played by William Hurt — and spent much of the movie feeling sorry for himself.

It doesn’t do newspaper publishers good to point fingers or see themselves as victims. They now need to move forward in constructive ways.

Related links and additional information:

• WSJ Blog
• LA Times Blog
• ZDNet

The Associated Press announced an aggressive new policy whereby it is going to try and get more money for newspaper content online or take legal action against “copyright violators” and those that won’t pay. From the AP’s own press release:

The Associated Press Board of Directors today announced it would launch an industry initiative to protect news content from misappropriation online.

AP Chairman Dean Singleton said the news cooperative would work with portals and other partners who properly license content – and would pursue legal and legislative actions against those who don‘t.

“We can no longer stand by and watch others walk off with our work under misguided legal theories,“ Singleton said at the AP annual meeting, in San Diego.

The quote at the end is vaguely reminiscent of the rogue US general in the 1964 Cold War satire Dr. Strangelove who will no longer allow the “commies” to “pollute our precious bolidly fluids.”

What’s unclear in the new AP policy is whether the organization would now want money for what I’ve just done: linking to a piece of AP content and quoting three sentences. It might.

While some of the interviews and discussions surrounding AP’s announcement suggest that the news association is trying to shut down wholesale or substantial copying of its articles without permission or attribution, there’s reason to believe that they will be going after the kind of use in this article as well. If they don’t want money, perhaps they’ll want advance permission to use the content.

Danny has written a very long and detailed discussion on his personal blog of why the AP policy is misguided, especially with regard to search engines (read: Google). Google has also responded on its Public Policy Blog:

We show snippets and links under the doctrine of fair use enshrined in the United States Copyright Act. The fair use doctrine protects transformative uses of content, such as indexing to make it easier to find [pdf]. Even though the Copyright Act does not grant a copyright owner a veto over such uses, it is our policy to allow any rightsholder, in this case newspaper or wire service, to remove their content from our index — all they have to do is ask us or implement simple technical standards such as robots.txt or metatags.

The Fair Use Doctrine says the following:

[F]air use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright. In determining whether the use made of a work in any particular case is a fair use the factors to be considered shall include —

(1) the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;

(2) the nature of the copyrighted work;

(3) the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and

(4) the effect of the use upon the potential market for or value of the copyrighted work.

The notion of what is “fair use” is somewhat ambiguous when applied to the Internet. It’s also at the center of this brewing controversy and may eventually get litigated by the AP. This would be a big risk for AP and newspapers more generally because a decision could go against them. To my incomplete knowledge the doctrine has not been tested in a web-related context around linking and excerpting content online. The AP maintains, however, this isn’t a “fair use” issue:

News aggregators and search companies have long asserted that collecting snippets of articles — usually headlines and a sentence or two — is allowed under the legal doctrine of “fair use.” News organizations have been reluctant to test that idea in court, and it is still not clear whether The A.P. is willing to test the fair use doctrine.

“This is not about defining fair use,” said Sue A. Cross, a senior vice president of the group, who added several times during an interview that news organizations want to work with the aggregators, not against them. “There’s a bigger economic issue at stake here that we’re trying to tackle.”

PaidContent’s interview with Dean Singleton sheds some light on the thinking behind the decision and announcement:

“I think our industry has been very timid about protecting our content, probably because we’ve done so well in the past few years that we didn’t recognize that misappropriation is as serious an issue as it is. As we’re now relooking at business models, it’s become clear that we must protect the rights of our content. … We perhaps have been timid about enforcing [those rights]. No more. We own the content but we’ve let those who spend very little, if any, get the most advantage from it.” What can they get from it at this point? “I am very confident that we will develop new models that help us get more.”

Newspapers want to exert more exclusive control over their content because they have lost faith in their ability to make money from advertising (read: search engine traffic) and will soon be trying to charge for access to it. Subscription and micro-payments models are being evaluated across the industry. Added Singleton:

“What’s becoming clear is that for many, many decades, advertising has supported our news mission and it’s not supporting it today. … If we don’t solve these issues, then survival is not assured. And we expect to solve these issues.”

So where do I start?

In fairness to the newspapers and AP there has been a lot of lifting of their content without permission, attribution or payment. Copyright holders do need to vigorously protect their interests or risk losing them. Having said that, the newspapers are partly hiding behind the copyright issue when, in reality, they have failed to compete successfully online. While they do create great content in many instances, they have lagged in innovating around business models and user experiences.

Danny suggests that the newspapers create “their own Hulu.” It’s a good suggestion. Long ago I also suggested that newspapers become news aggregators themselves. Why didn’t the newspapers invent Digg or Yelp or Trulia? Though there were and are some interesting experiments going on across the industry, there’s something in the culture of newspapers, it would appear, that has effectively inhibited them from fully embracing the Internet and new thinking about their content — beyond putting it up online. In fairness, there’s also a resource issue too.

While I acknowledge the right of newspapers to control their content, there’s a way in which this policy flies in the face of the culture of the Internet. It’s consistent with the worldview and approach that has landed newspapers in the predicament they’re in today. On some level it seeks to turn back the clock.

And while it’s true hypothetically that if there never was a Google News or Yahoo News or Topix (itself owned by newspapers) perhaps people would be going directly to newspapers sites to get their news and other local content. Yet the nature of the Internet made these aggregator sites all but inevitable. Why didn’t the newspapers develop them first?

I acknowledge the dire predicament newspapers are in and that they need to “do something.” (I’ve devoted lots of writing on my blog to newspapers.) And honestly I wouldn’t have any magic solution if I were being asked to devise a strategy to “save the industry” at this point. However they should seek to exploit distribution — including mobile and its new opportunity to charge for content — rather than limit it.

There’s also a way in which this newly announced policy is a giant PR blunder for AP and by extension newspapers. They might have consulted with Google and other “aggregators” and negotiated a more online-friendly policy before going public with an approach that feels both hostile and desperate.

___

Postscript: Google’s Eric Schmidt addresses the NAA (newspaper) conference (audio).

Newssift, a semantic business news search engine created by The Financial Times, has launched in beta tonight. Currently indexing thousands of news sources worldwide, and with millions of articles in its database, Newssift aims to bring context and meaning that its creators say is missing from traditional keyword-based business news search engines.

“We want to put an angel on the shoulders of the business person,” says Newssift CEO Robin Johnson.

We took Newssift for a pre-launch spin, and the search engine does offer a compelling experience as a business research tool. The screenshot below shows Newssift’s results on a search for “Internet advertising.” Scroll down for an explanation of what you see.

Users can refine queries across multiple search options, such as Topic, Organization, Place, Person, and Theme/Keyword (see the gray area at top). Below the suggestions, Newssift’s search results include additional context, such as the ability to refine results by Sentiment and/or Article Sources (see pie charts on left). Using the screenshot above as an example, a searcher could quite easily drill down to find news articles that talk positively about Yahoo and its new CEO, Carol Bartz … and then click to see negative coverage, too. The ability to save searches is a nice feature, as is the saving of your search history, both of which can be accessed quickly from the search engine’s main navigation bar.

What Newssift hopes will separate itself from other news search engines — and business news, specifically — is its ability to add meaning and context to news search results. Johnson says Newssift is the first search engine to apply a relationship-based algorithm to business news. Newssift wants users to begin with generic queries, and use its refinement technology to find the most relevant results.

A team of editors helps choose and aggregate the sources that Newssift crawls, but Johnson says the engine gives no preference to Financial Times news articles. Indeed, on a purely anecdotal level, I rarely saw Financial Times articles appearing in the results of the dozen or so queries I did. (The only prominence FT gets on Newssift is that it’s separated as a sub-option under Newspapers in the Article Sources refinement — see above.)

With Newssift now in its public beta, the creators are looking for user feedback and say they’re planning to expand the source list by crawling more online news sites, including blogs.

MeeHive, a personalized service that scans more than 5,000 news sources and 5 million blogs to bring you fresh news and information based on your interests, launches today. MeeHive organizes information “relevant to a person rather than a topic,” with “fundamentally time-changing information,” said Kosmix co-founder Venky Harinarayan.

The service is very easy to use. You can select general topics of interest by ticking category boxes such as “online advertising,” “food and wine,” or “gadgets,” or more specific topics by typing in search queries that are saved as your own custom categories. Your personal page is then tailored to display a range of content based on your interests, and is updated daily.

MeeHive also lets you track your favorite news feeds, and also lets you search for your friends social activity on the web by typing in their names or email addresses. There’s also a section that tracks hot Twitter topics.

A mobile version of MeeHive is available on the iPhone, either by logging in on the iPhone’s web browser, or via a downloadable app from the iPhone app store.

The word “content” wasn’t historically used to broadly describe writing or imagery or video until the internet. The term “content” also implies a kind of generic equivalence — lots of things that are more or less of equal value or can be lumped into the same bucket.

That notion certainly applies to what’s happened to news online. We all know the well-documented survey data that show the internet has become the preferred news source (or in some surveys second to TV) across age groups in the US. We all also know the terrible financial condition of major metro daily newspapers in the US. The two are causally tied together.

The myriad news sites, aggregators, feed readers, as well as search engines and portals, have turned news over the past several years into something of a “commodity.” That didn’t used to be the case, with various new outlets operating in a kind of hierarchy of credibility. While it’s untrue to say there are no more news “brands,” the internet has “democratized” or flattened news, exposing a wide range of sources of coverage and making them easy to access.

It is in this context that, under the inflammatory headline “Google Devalues Everything It Touches,” Tom Foremski summarizes a discussion on the Charlie Rose show about the plight of newspapers and the role of the internet in that predicament featuring executives and editors from Time, the Wall Street Journal and the New York Daily News.

Foremski quotes Robert Thomson, Managing Editor of the Wall Street Journal:

But one of the — Google — I mean, the harsh way of just defining it, Google devalues everything it touches. Google is great for Google, but it’s terrible for content providers, because it divides that content quantitatively rather than qualitatively. And if you are going to get people to pay for content, you have to encourage them to make qualitative decisions about that content.

Even though Google has considered ranking news stories by their “authority,” the remark above is essentially correct — brand per se doesn’t play into the presentation of news content in Google search results or Google News. (Google is a stand in here for “the internet” in most of the discussion.)

In the past, I’ve observed that search is something of a “brand killer” because of the way that content is presented in search results. However Yahoo Search Monkey, interestingly, attempts to give publishers control the presentation of their content in search results and offers them some “branding” in the process.

For its part Google has been a big booster of traditional newspapers and their value, somewhat ironically. It has commissioned studies to prove their value. Eric Schmidt has publicly spoken many times of the importance of traditional newspapers. And Google’s Print Ads program, recently discontinued, was touted as a way to help grow ad pages in traditional papers.

It’s also important to point out that it’s not simply the flattening of news or the way that news is presented online that has negatively impacted newspapers and the value of their brands. It’s also the emergence of myriad alternative online content sources and tools that deliver what newspapers used to deliver almost exclusively — such as restaurant reviews or event and entertainment listings. And then there’s the more efficient delivery of advertising online. Before Google became the juggernaut it is today online jobs, autos and real estate destinations were already starting to capture traditional newspaper advertising.

Foremski argues that US newspapers can take lessons from British newspapers, which have better “exploited” the internet for years. Maybe that’s true in a relative sense but UK newspapers are not immune from the same forces that have resulted in today’s US newspaper crisis. As a result US newspaper publishers are talking again about charging for digital access to their “content.” That’s going to be a challenge; however it’s probably necessary if they’re going to survive.

TweetNews is a new search engine that uses hot Twitter topics to bring more relevance and freshness to news search. It’s a pet project developed by Vik Singh, a member of Yahoo’s BOSS team.

As Vik describes it on his personal blog, fresh news often doesn’t have enough links to rank well in a traditional algorithm. But TweetNews replaces links with Twitter conversations as a signal; news stories that are being talked about the most on Twitter will rise to the top of a TweetNews search.

It’s an interesting mashup because it plays to the real strength of Twitter — its ability to report and spread news before traditional news sources, and its ability to serve as a barometer for what’s “hot” or important at any given moment. A great example happened just today, where one of the earliest and most remarkable photos of the US Airways crash in New York City was first published via Twitter.

Vik says his inspiration for TweetNews came from doing news searches during the recent California wildfires and the terrorist attacks in Mumbai:

What I found most interesting in both of these cases was that news articles did exist on these topics, but just weren’t valued highly enough yet or not focusing on the right stories (as the majority of tweets were). So why not just do that? Order these fresh news articles (which mostly provide authority and in-depth coverage) based on the number of related fresh tweets as well as show the tweets under each. That’s this service.

As he mentions above, TweetNews not only uses Twitter to determine relevance, but it also shows the social commentary about each story that’s happening on Twitter. It’s a good advertisement for Yahoo’s BOSS, although, in a bit of irony, it runs on Google’s App Engine.

The code is open source. It’s available, along with more background and screenshots, on Vik’s blog.

How would you like to track hot news geographically? Or see where social conversations are happening on a map? Maybe you’d like to find businesses that are open 24 hours a day, or research keywords on a state-by-state basis. If so, read on for a look at five search tools you may not know about, but should.

About two months ago, we published 7 Search Tools You May Not Know … But Should. That became a popular article and let us know that we should continue uncovering largely unknown search tools in future articles. So, here’s the second in an irregular series of posts about search tools you should know about.

Track This Now

Track This Now is a news search engine that displays results geographically on a Google Maps interface. Track This Now offers real-time tracking of news articles from 236 countries around the world. Just provide the topic you want to search for and the country(ies) you want to target, and Track This Now shows you news coverage of that topic mapped out. Here’s a recent search for “Google” targeting the U.S.

According to the web site, a widget is in development to let you embed news search maps directly into your blog or web site. And Track This Now also recently added geographic search engines for tracking any keyword/topic on Twitter, Flickr, and YouTube. All in all, the suite of search tools at Track This Now offer a compelling way to get a snapshot of what’s going on now in news, social conversation, images, and videos.

StateStats

This is a search tool that shows the popularity of any search term on a state-by-state basis in the U.S. For example, StateStats shows that states in the southeast U.S. tend to do the most searches for the term walmart.

You can see the state rankings on the map at left. The second column adds extra information about other terms that correlate with your original query. Creator Doug Beeferman specifically warns against drawing certain conclusions from the data in the second column:

“… the fact that walmart shows a moderate correlation with ‘Obesity’ does not imply that people who search for ‘walmart’ are obese! It only means that states with a high obesity rate tend to have a high rate of users searching for walmart, and vice versa.”

All disclaimers aside, this is an interesting search tool that mashes up search data from Google Insights for Search with state data from the U.S. Census Bureau.

Jogli

Jogli is a music search engine that claims to have more than 12 million albums and 500 million songs on offer, and you can listen and watch videos for free. A search for your favorite band brings up a pretty standard interface, showing albums and videos.

Below the fold are sections showing related artists, playlists created by other users, and more. When you search for an individual song, the interface changes slightly to show lyrics prominently. It’s easy to use and runs pretty deep — it found a little-known UK band I like called Vib Gyor, but didn’t find a little-known Irish band I like called Agiven. If anything, Jogli seems to rely too heavily on sourcing videos from YouTube, which becomes a problem when things happen like Warner Brothers pulling videos from YouTube.

2itch

2itch is a local search engine with a unique twist: Its goal is to list businesses that are open 24 hours a day, and show them on a Google Maps interface. 2itch claims to have more than 4,000 U.S.-based, 24-hour businesses listed. A search on my semi-remote zip code only brings up a few truck stops along the highway in an hour or two in each direction; the best coverage, understandably, is in a handful of major cities: New York, Los Angeles, Chicago, San Francisco, and Phoenix.

There’s an add listing form where users can submit businesses to be listed. 2itch already offers a Windows Vista Sidebar Gadget and a mobile version; the site says an iPhone app is in development.

Ninja Tickets

In the previous tools column, we profiled a new search engine for tickets called FanSnap; in this column, we’ll profile an older ticket search engine called Ninja Tickets. Like other ticket search engines, Ninja Tickets doesn’t sell anything — it aggregates tickets that are available through other ticket-selling web sites. What sets Ninja Tickets apart is its Price Rating tool, which attempts to list tickets in the order of best market value based on price and seat location. The Price Rating tool appears as a star-based rating next to each search result.

Ninja Tickets does caution, though, that their search engine is still in beta and that users should check to make sure tickets don’t have obstructed views.

Finally, an update…

Soovle, one of the search engines covered in the previous article, has added a few new features based on user requests from new traffic after being profiled on Search Engine Land. Chief among these is the ability to store, send, download, or print the keywords they discover via Soovle. This is done by simply dragging keywords to a new book icon in the upper left of the Soovle interface.