A lot of the innovation in search these days is all about What’s happening right now? Twitter is the poster child where real-time search is concerned, and despite its ongoing spam problem, Twitter Trends has become one of the primary places to get a snapshot of what’s happening now.

Google Blog Search is getting in the game with its announcement today of “Hot Queries,” a new addition that appears in the upper right corner of the Google Blog Search home page. It shows 10 queries that are “happening now” — or, “currently popular” in Google’s words — in Blog Search. What we don’t know is if the list is ranked in actual order, or randomized; and we don’t know how often the Hot Queries list is refreshed.

That’s not the only new addition to the Google Blog Search home page; right below Hot Queries is a section called “Latest Posts,” which shows (and links to) 10 recent posts from “popular blogs.” Google hasn’t given any indication how it chooses which blogs to feature in this space.

Perhaps the most utilitarian new addition to Google Blog Search is RSS and Atom feeds that let users subscribe to any topic or story. Links for those are in the left column.

Lastly, there’s also an iGoogle Gadget that lets users embed the Google Blog Search home page on their iGoogle page.

There’s no official announcement (yet), but Google tells Search Engine Land that it’s made several improvements under the hood of Google Blog Search.

In an email conversation, Google’s Jeremy Hylton — head of the search quality group in Google’s New York office — explained how the changes impact the “clusters” of stories that appear on the blog search home page.

“We’re doing a better job of choosing the blog posts to include in clusters,” Hylton says. “We’re also working on changes to expand the number of posts we consider for clustering.”

One of the algorithmic changes is aimed at making sure the home page clusters reward the freshest and most authoritative blog content. Within any cluster, Google wants to find the posts that people are talking about the most.

“We have a lot of ranking signals. We’ve been tuning the ranking and clustering algorithms to make better use of those signals. One important changes is that we’re processing new links much faster, so the post that breaks a story and gets a lot of links is more likely to become the lead story.”

The update also includes some minor design changes. Gone are the blue boxes to the left of each cluster, as shown here. The new layout seems more spartan (”streamlined,” in Hylton’s words):

But Google’s changes are more about what powers Blog Search, not what it looks like.

“With [Thursday's] update,” Hylton says, “the whole collection of ranking changes provides a more authoritative set of results. I think it’s really a great set of bloggers talking about current events.”

Hylton says they’ll continue tweaking the algorithm and interface moving forward to improve the overall user experience.

While Twitter is looking at ways to be more search-ey, Google is apparently thinking about ways to be more microbloggy. Buried in a Reuters article about Google’s shareholders meeting yesterday is this paragraph about the company’s interest in the social web:

“Company executives who appeared alongside Schmidt at the media briefing said Google was looking at ways to generate money from the surge of social networking activity on the Internet, as well as at ways of integrating microblogging capabilities, such as those popularized by Twitter, into its search product.”

(emphasis is mine)

It’s not in quotes, so we’re left wondering what the exact words and context were … not to mention how Google might “integrate microblogging capabilities” into search. That seems to suggest something more than just offering “Twitter search” or improving its real-time search capabilities.

On ReadWriteWeb, Marshall Kirkpatrick speculates three possibilities: adding microblogs as a search option (like images, video, etc.), using microblogging links to supplement the freshness factor of Google’s algorithm (which I suspect is already happening to some degree), or adding a status update box to its search interface, perhaps encouraging users to answer the question, “What are you searching for?”

Marshall thinks that last option is most likely, but I’m thinking a version of the first idea is most likely: not just adding a microblog search option, but adding real-time/microblog results as a onebox/universal search result. There’s already a Greasemonkey script that does this. But just as Google imports news articles, blog posts, videos, etc., for its blended search results, I could see microblog updates being pulled into Google’s search results. This would make the most sense on hot topics, breaking news, and the more immediate types of queries/information where Google’s traditional results don’t always give the user the information s/he wants.

Then again, I never thought I’d see the day that Google would let users add or remove search results, so take my guess with the proverbial grain of salt.

Update: On the L.A. Times blog, David Sarno shares the exact text of what Marissa Mayer said yesterday about Google and microblogging:

“What’s really happening in Twitter is that there are a lot of clues in it in terms of what’s happening that’s interesting overall. It’s similar to what we see in Google Trends, where people will often type what they’re interested in into the search box, and we can make some predictions off of that.

So we are interested in being able to offer, for example, micro-blogging and micro-messaging in our search. Particularly in Blog Search and possibly in Web Search, but we don’t have any particular plans to announce.”

So, taking that at face value, it sounds like we shouldn’t expect Google to be adding any update-your-status boxes, but a more obvious integration of Twitter/microblogging content into search results is on the table.

Postscript by Barry Schwartz: Google Operating System spotted that in the Google in Your Language project, Google is asking translators to help translate the description of a possible Google Microblogging search engine. The specific text Google is asking to be translated is:

Recent updates about QUERY. This is the MicroBlogsearch Universal result group header text. A Microblog is a blog with very short entries. Twitter is the popular service associated with this format.

Just some more evidence that Google is working towards a Google Micro Blog search engine.

A couple weeks ago, we reported that Google fixed the blogroll indexing algorithm in Google Blog Search, but has not yet fixed the link operator, which returned links found in blogrolls. Google has recently updated the link operator for Blog Search to not count links in the blogroll for the link operator command in blog search.

As I reported at the Search Engine Roundtable this morning, Jeremy Hylton of Google said Google Blog Search “now drops most or all of the links that occur in the blogroll or in other parts of the page that are just boilerplate.”

This issue dates back to early November when Google Blog Search began indexing full text of the blog post, including, in some cases, the blogroll and boilerplate portions of the site. Over the course of now and then, Google has been working on tweaking these detectors to unique content and pushed out a major update a couple weeks ago and now has updated the link operator (i.e. link:www.example.com).

My early tests show very positive results, but if you find any issues, please let Jeremy know in this Google Groups thread.

As Barry Schwartz points out today on Search Engine Roundtable, the Google Blog Search team is rolling out fixes to how it separates blogrolls from actual blog posts.

Back in December, we wrote about Google’s switch to full-text indexing for Blog Search, but that led to problems identifying blogroll links as part of blog posts.

Barry points to a Google Groups discussion, where Google’s Jeremy Hylton says:

“We have launched a ranking change that reduces the number of results that are returned because of blogroll matches. There are still problems to work out, but this change appears to be a big improvement over our earlier fix.

So, it sounds like the problem isn’t fully solved yet, but they’re making progress.

There’s a lot of stats and analysis today about Twitter’s traffic, how it compares to other social search sites, and who’s benefiting from all the traffic Twitter can send. It seems no one argues that Twitter is on the rise, but just how high it’s risen is still up for debate.

Let’s start, appropriately, with a tweet: comScore posted a note that Twitter’s February traffic was up 55% over January. (We don’t see a formal release of this data yet, but will update when we do.) 55% is nothing to shake a stick at; it’s serious growth.

Steve Rubel dug into Twitter Search traffic (i.e., to search.twitter.com) today, and suggests that it’s about to surpass traffic to Google Blog Search:

Steve’s numbers come from Compete.com, which show Twitter Search getting 1.35 million visitors per month, slightly less than Google Blog Search’s 1.38 million. But other data services don’t show the same thing. Here’s how Quantcast — a service that some believe is generally more accurate — compares Twitter Search and Google Blog Search:

There’s no way to know which is correct without seeing actual data from Twitter and Google (don’t hold your breath…), so it’s anyone’s guess if Twitter Search is close to surpassing Google Blog Search. But from looking at the charts above, it doesn’t seem at all farfetched to think it will happen at some point in the future.

Meanwhile, Hitwise’s Heather Hopkins was also writing about Twitter today, but more specifically about where people go from Twitter.

Somewhat surprisingly (to me, at least) the Hitwise data shows Google as the largest recipient of Twitter traffic … and no, that doesn’t include traffic to YouTube (which I thought would be much higher than seventh):

Another thing worth mentioning is Heather’s comparison of traffic categories. She says Twitter’s clickstream doesn’t send as much traffic to shopping and educational sites (like Wikipedia) as search engines do. It sends more traffic to other social network sites and to personal blogs/web sites than search engines do.

In that sense, Twitter itself behaves just like you’d expect it to — like a social network. But the growth of Twitter Search, as evidenced by the charts and data above, suggests that Twitter is moving into a different realm, somewhere between social networking and search, and perhaps bringing together the best of both worlds. Social search, as Steve Rubel says in his piece today, “adds a much needed layer of trust to traditional search that helps us qualify sources.”

One of the requirements of class-action lawsuits is notification of potential class members. Most of the members of a class never know that a lawsuit has taken place on their behalf (e.g., bank X credit card holders). So the courts require attorneys to tell class members of the suit and settlement terms and allow them to opt out of the settlement if they desire. If they fail to opt out typically they’re bound by the terms and cannot sue independently.

Google is now in this position with the settlement of the Google Book Search Copyright Class Action lawsuit against its book scanning project. As part of the settlement, Google has set up a $125 million fund to settle claims, which is discussed on this dedicated website constructed to report the settlement terms and related rules.

To satisfy the notification requirements, in addition to the settlement website, Google has used a variety of methods to communicate with potential claimants. According to an article in the NY Times, it has used direct mail but it also plans to spend millions ($7 -$8 million) on newspaper and print magazine advertising to alert copyright holders about the settlement. The scope of all this is global — hence the price tag.

I would be remiss if I were not to point out the partial irony here: Google using traditional advertising to promote an online initiative. But the Internet, especially outside the West, still arguably doesn’t have the same reach as traditional media, which are thus critical to communicate with copyright holders, authors and publishers. (As an aside, if newspapers in certain US cities disappear it will be difficult to find a judicially acceptable alternative notification mechanism — perhaps a combination of TV, Internet and direct mail.)

The way the Book Settlement ad spend is apparently being distributed, according to the Times’ article, is 30 percent US, 30 percent “industrialized countries” and 40 percent “rest of world.” This apparently reflects the regional contribution of books to the global library.

It has been almost three months since Google Blog Search began indexing full text of the page and, since then, there have been numerous complaints from searchers and webmasters.

The majority of the complaints are that Google Alerts are coming up for irrelevant or outdated blog results. In addition, many blogs are no longer indexed quickly anymore. Plus, Google is indexing blogroll code and including them in the link command results.

Jeremy Hylton from the Google Blog Search team updated the Google Groups thread saying that Google will be conducting “visual experiments early next month” that will start with the link: queries and focus on “blogroll detectors” in the matching algorithm. Personally, I am excited to see improvements with Google Blog Search.

Why do I care? Well, I care for a few reasons. As a searcher, I love being able to find relevant and fresh blogs with content on topics that I am looking for. I know that I am missing key blogs in my discovery process and I can’t wait for the results to become fresher, quicker, and more relevant. Yes, I know, easy for me to say, but hard for an engineer to make happen – good thing I am not a Google engineer.

In addition, I often track who is linking to the articles I write. This way I can see what people are saying about my stories and help clarify my stories when necessary. I often do that by watching Google Blog Search for queries on names such as Barry Schwartz, Search Engine Land, Search Engine Roundtable or RustyBrick. I also use the link operator to find out who is linking to my stories on either Search Engine Land or the Search Engine Roundtable. And ever since November 2008 the link operator in blog search failed for me. Then it got worse when they tried fixing it and got somewhat better with their second attempt towards the end of December.

I know many searchers, webmasters, and SEOs who would love to see future improvements. So I hope this early March update will make for a better Google Blog Search, as Jeremy said it might.

It’s been about two months since Google Blog Search was relaunched with a new front page that summarizes stories. I talked with Google more about some of the inner workings at the end of October and finally am getting around to posting this, spurred by one of the planned changes becoming official. Google Blog Search now uses the full-text of posts (in most cases), rather than using whatever was in a blog’s feed (which could often be only part of a post).

It was always annoying that Google Blog Search only depended on what was put out in a feed, rather than actually indexing the full-text of a blog post. Some publishers don’t put out full-text feeds (like here on Search Engine Land) for a variety of reasons, including the fact that putting out a full-text feed often is misinterpreted by others as a right to reprint a post in its entirety without getting formal permission.

Problem solved! Vanessa Fox’s recent Google Blog Search Changes How It Indexes Posts article covers how Google’s now officially confirmed for the first time that blog posts are being spidered. From what Google posted to its support groups on the subject:

We have changed the way we index blog posts to include the full content of the page. We’ve had occasional complaints about the use of the feed content, particularly the problem with partial feeds that you mentioned. The indexing change has improved the results for a lot of queries, both because we have the full content of the page and because we extract links that are missing from the feeds.

It’s a welcome change. When I talked to Google about it in October, they said it was in the process of being rolled out slowly. Now it is fully live for all new posts that get into Google Blog Search. However, some older posts may not be fully indexed. Google expects that by early 2009, all pages indexed by Google Blog Search since June 2005 — which it said are in the billions — will be fully indexed.

And how do you get in Google Blog Search again? If you’re not already there, the easiest way is to “ping” the service with your blog’s home page or your blog’s feed. You can use this submission form. Better, your blog should automatically ping Google each time a post goes up. Most blogging software is either already enabled to do this or it’s simple to add it. But Google provides further instructions here.

In the past, pinging made Google simply grab the latest post as shown in your feed, which is why those putting out only partial feeds did not have their posts fully indexed. Now, a ping is supposed to cause Google to immediately grab the full text of your post (I haven’t tested this yet, however). If it works as advertised, that means within seconds, your full post should be indexed and searchable within Google Blog Search.

A downside to full-text indexing is something that Barry Schwartz noted earlier, as full-text indexing got rolled out unannounced over the past few weeks. Blogs often have blogrolls, links to other blogs. Now that full-text indexing is being done, links from these blogrolls caused some people to think there were new blog posts being done about them. From what Google also posted in its groups about the issue:

The downside of this change is that we see more results that match only the blogroll and other parts of the page that are common to all of a blog’s posts.

We expected some problems from blogroll matches, but may have underestimated the impact on searches using the link: operator or where the query matches a blog or blogger’s name. We do expect to fix the problem you’re seeing. We’ll use the full page content, but exclude the content that isn’t really part of the post. I’m not sure if we’ll be able to make the change before the end of the year, but we are working on it and are pretty confident that it can be solved. We’ll post an update here when we’ve got a solution.

Google also just sent me this update:

Yes, we’ve got a change that should help with the blogroll issue. Although, it’s not 100% perfect, it performs quite well in our tests. We deployed it for link: queries yesterday and hope to have it deployed for all queries in the next few weeks.

Meanwhile, what about how things are going with the new front page to Google Blog Search? One issue that was plaguing Google Blog Search soon after launch was that spam blogs seemed to be getting featured and sometimes embarrassing play within the service.

Jeremy Hylton, a software engineer who works on the Google Blog Search project, said that Google was aiming to get more reputable blogs featured, that Google Blog Search has an internal ranking of blogs by quality and authority that it could use. Looking today, things seem pretty clean — so I assume they’ve notched up the standards.

A confusing thing to me after Google Blog Search launched was the order story “clusters” or “groups” are listed on the home page as well as the subject-specific pages, such as Technology. Each group has a box that shows you the number of blogs estimated to be discussing a topic and the time period where the discussion is stretching over:

The screenshot above shows how a story about Walmart rumored to be selling iPhones has been mentioned on 41 blogs over 12 hours. Now consider what comes below it:

See how under the iPhone story there’s a another story group about a Google Android phone being released in Korea. How come the iPhone story mentioned on 41 blogs over 12 hours trumps the Android story with 57 blogs mentioning it over 18 hours?

Hylton said that part of the ranking process is to look at the “burst of activity” around a story. For example, a story with a sharp spike in mentions that’s recent compared to other stories might come higher on the list, being deemed newer news.

To see some of the spikes, check out any story’s “cluster” or “group” page. I keep using quotes because Google itself doesn’t have a name for these. But you get to them by clicking on the green link showing the number of blogs talking about a particular story. Here’s one for the Walmart iPhone story that I mentioned. On it, you can see a chart on the left showing the number of blogs covering the story and when those stories were spotted:

As a side note, while each story cluster/group has its own summary page, those URLs sadly don’t seem to remain over time. I had examples from things we discussed back in October like this cluster which now simply resolves to the Google Blog Search home page. I wish the URLs would continue working permanently.

Also related, watch for those story cluster pages to be enhanced over time. Hylton said that Google’s looking at ways they can build out more context about a particular story.

When Google announced its new, clustered results on the Google Blog Search home page, they promised support for more languages “in the coming months.” Well, less than two weeks later, the clustered home page has launched on Google China Blog Search.

This is the first non-English launch of the new blogsearch interface. Google China Blog Search covers just Chinese-language blogs. As Google promised a couple weeks ago, expect more launches of the new blog search home page in the future.