The Wall Street Journal reports that the US Department of Justice (DOJ) has sent formal requests (”civil investigative demands”) to book publishers and Google regarding particulars of their book search settlement deal. That settlement has drawn considerable criticism from various third parties. The article characterized the requests as an “escalation” of the DOJ’s probe:

A New York publishing executive said the Justice Department is requesting documents about pricing, digital strategy and conversations with other publishers related to the Google settlement. “The Justice Department is clearly focused on Google,” said this executive. “It’s a wide-ranging request for documentation.”

The article then expands into a wider discussion of anti-trust concerns and questions surrounding Google, which is seeking to portray itself not as the king of search but as a smaller player in a much larger game — all advertising.

In a new more regulation-friendly political climate, Google will be both stepping up its lobbying and facing more scrutiny from legislators and the DOJ on every move. As a consequence we’re unlikely to see any blockbuster M&A activity from Google for the near future at least.

Profy reports that a Russian company is suing Google for $3 billion over Google’s contextual ad program, AdSense.

The Russian company, Era Vodoleya, claims they patented contextual ads a year before Google came out with AdSense. The funny thing is this is not the first time Google was sued for patent infringement over AdSense. We reported back in August that Daniel D. Wexler is also suing Google for the same thing. Anyone else want to take shots at Google for AdSense?

Profy suspects that the Russian company is simply looking to “blackmail” Google and settle out of court for a nice fee. It will be interesting to see if Google does settle or if they fight it in court.

It’s fairly safe to say: 1) patent law is a mess, 2) patent law is in transition, and 3) newer court rulings are reducing the rights of patent holders. But in a very provocative argument, based on recent cases, the PatentlyO law blog argues that new rules and tests imposed by courts (though not yet the US Supreme Court) could effectively eliminate software patents.

The article uses Google PageRank as the example, but the logic would equally apply to all software patents. Here’s the critical summary of the new rules: In a series of cases including In re Nuijten, In re Comiskey and In re Bilski, the Patent and Trademark Office has argued in favor of imposing new restrictions on the scope of patentable subject matter set forth by Congress in §101 of the Patent Act. In the most recent of these three—the currently pending en banc Bilski appeal—the Office takes the position that process inventions generally are unpatentable unless they “result in a physical transformation of an article” or are “tied to a particular machine.”

Software essentially wouldn’t qualify under these tests. It wasn’t until the 1981 that software could be patented at all. Prior to that the US Patent & Trademark Office refused to offer patent protection to “mathematical algorithms” (computer software).

It’s important to note that none of the cases discussed above are US Supreme Court cases and thus not ultimately determinative of whether software patent protections will be effectively voided. I would imagine that the Supreme Court would not go as far as the Patently O article implies. However, the Court does seem quite willing to restrict the scope of patent protection and has done so recently, for example, in the case of Quanta Computer, Inc. v. LG Electronics, Inc., decided in June of this year.

As the article suggests, the combined impact of these cases may be radical and effectively void all or most software and algorithm patents. That would be a very mixed blessing for everyone.

I don’t believe it will get to that point (though I’m not a patent expert), but we’re clearly in an era where the US PTO and federal courts are trying to reign in and clean up what they perceive to be an unwieldy system.

Digg And Others Sued For Infringing Infamous Computer Solitaire Patent from Techdirt reports Google, Yahoo, AOL, and even Digg were all sued by Sheldon F. Goldberg for patent infringement of US Patent 6,264,560.

The Electronic Frontier Foundation calls Sheldon a wanted person who has crimes against the public domain for threatening small online gaming websites and claiming to own basic online gaming architecture. He has been using these patents to threaten many companies but has now actually filed lawsuits against Google, Yahoo, AOL, and even Digg. The patent covers playing games like solitaire online.

U.S. appeals court hands Google patent case setback from Reuters reports a US appeals court has partially overturned the initial ruling in Google favor in a patent infringement case filed by Hyperphrase Technologies, LLC.

The court supported the initial ruling that AdSense did not infringe on Hyperphrase patents, but it did not uphold the initial finding that the autolink feature in the Google Toolbar was not a patent infringement. Instead, it sent that portion of the case back to a Wisconsin district court for reexamination.

The official decision can be read in this PDF file.

Northeastern sues Google over patent from The Boston Globe reports that Northeastern University and a company founded by a Northeastern professor, Kenneth P. Baclawski, is suing Google for patent infringement.

The issue at hand in this case is that Northeastern has patented a method of storing database queries back in 1997. It wasn’t until about two years ago that Northeastern learned that Google was potentially infringing on that patent. It wasn’t for another couple years that Northeastern was able to find lawyers that would take the case on a contingency basis.

The complaint was filed on November 6th at Eastern District of Texas–the U.S. court. You can find the PDF of the complaint over here, as well as the copy of the patent and the court docket, thanks to ResourceShelf.

In addition, you can read about how Microsoft does index partitioning in their search index via SEO By The Sea.

AOL, Google, Microsoft, Yahoo Sued Over Competitive Bidding Patent from InformationWeek reports Performance Pricing filed a patent lawsuit (PDF document) against AOL, Google, Microsoft, and Yahoo over how they use competitive bidding within their search ad marketplace.

The patent, named Systems and methods for transacting business over a global communications, was granted in 2005, and covers actives such as “video games, electronic board games, crossword puzzles or other word games, sports betting, card games, or any other activity or combination of activities,” InformationWeek reports. The patent holder feels it also covers the bidding methodologies deployed at Google AdWords, AOL Search Marketplace, Microsoft adCenter, and Yahoo Search Marketing.

Until very recently, you might have seen a label next to a search result in Google that indicated it was a “supplemental” result. A couple of patents from Google, one of which was granted this week and one from earlier this year, discuss how a search query might return results from an extended database that sound a lot like a supplemental results.

The Official Google Webmaster Central Blog announced that they would stop labeling their supplemental results in a post from July 31st, titled Supplemental goes mainstream. The authors, Prashanth Koppula and Matt Cutts, tell us that the system for crawling and indexing supplemental results has been improved, and those results are fresher and more comprehensive than ever.

Danny wrote a detailed post on the same day – Google Dumps The Supplemental Results Label

The patents may provide some insight into how a supplemental or extended index works, and how partitions are used to speed up a search of extended results.

These patents don’t use the word “supplemental” but it is possible that they describe the way supplemental results work, or worked. In the Webmaster Central post, we are told that supplemental results were introduced in 2003. These patents were also originally filed in 2003.

What I found interesting about them is that they provide a view of how indexing could work in a search engine, and answer some questions such as: (1) when are extended database results triggered, (2) how search result numbers are estimated, and (3) why you sometimes see a link at the bottom of results that tell you there are more results that aren’t being shown, that you can see if you click upon that link.

Some information about the patents:

System and method for selectively searching partitions of a database Invented by Kourosh Gharachorloo, Fay Wen Chang, Deborah Anne Wallach, Sanjay Ghemawat, and Jeffrey Dean Assigned to Google US Patent 7,254,580 Granted August 7, 2007 Filed: September 30, 2003

Abstract

When a search query is received, a plurality of partition indexes are searched using the set of search terms in the search query. Each partition index corresponds to a partition of a document index. The search of each respective partition index identifies a subset of a plurality of document index sub-partitions corresponding to the respective partition index. Next, the search query is executed by only those document index sub-partitions identified by the subsets, thereby identifying documents that satisfy the search query. By using the partition index to reduce the number of document index sub-partitions searched while executing a search query, the execution of the search query is made more efficient.

System and method for searching an extended database Invented by Kourosh Gharachorloo, Fay Wen Chang, Deborah Anne Wallach, Sanjay Ghemawat, and Jeffrey Dean Assigned to Google US Patent 7,174,346 Granted February 6, 2007 Filed September 30, 2003

Abstract

Once a search query is received from a user, a standard index is searched based on the search query. The standard index forms part of a set of replicated standard indexes having multiple instances of the standard index. A signal is then determined based on the search of the standard index. When the received signal meets predefined criteria, an extended index is searched. The extended index forms part of a set of extended indexes having at least one instance of the extended index. There are fewer instances of the extended index than instances of the standard index. Extended search results are then obtained from the extended index and at least a portion of the extended search results is transmitted towards a user.

What follows is a walk through of the process of returning search results from the standard index and the extended extended index, when necessary. Some of the alternative approaches mentioned in the patents aren’t covered or discussed in detail. The processes described may be very different from the reality, but hopefully this view of an extended index will provide you with some insights into how documents could be indexed by a search engine, and give you a slightly different perspective on the process of returning results in response to a query.

Searching the cache and standard index

A searcher submits a query to the search engine, and the query is received at one of a number of datacenters and sent to one of the query servers at the datacenter.

The query server receives the query and sends it to a mixer. The mixer transmits the query to the cache, to search the cache for results. The mixer might first normalize and hash the search request.

A hash value representing the query is received by the cache, and the cache is searched.

If a match for the hash value is found, those results would be sent back to the mixer. Results might be a list of located documents, with or without snippets, or an indication that there were no results in the cache.

The mixer or query server receives that response and determines whether results were located. If there are results, and snippets weren’t returned with them, they may be requested from the cache, and if they aren’t in the cache, they might be requested from the standard document server.

If no results were located, then the query is sent to standard index server. The search request could be first transmitted to multiple standard balancers (one within each partition) that transmit the search onward to the standard index server.

Each balancer transmits the search request to a set of standard index servers.

Each standard index server stores and searches one or more partitions of the standard index to produce a set of search results. Each balancer may send the search query to between ten and one hundred standard index servers, and each standard index server is set up to store and search multiple (e.g., two to ten) index sub-partitions.

When the query is received by the standard index servers, those are searched, and the results are sent back to the mixer. Those results could be a list of located documents or an indication that no results were found.

The mixer receives a response, and if no search results were located, notifies the searcher that there were none.

If search results were located, snippets might be requested from the standard document servers, or the results might be sent to the query server, which might request the snippets.

The standard document servers receive that request for snippets, generate them from the documents identified in the search results, and send the snippets back to the mixer.

The mixer then sends the results and snippets to the cache, where they are saved in memory for future searches for that query.

At this point, a decision needs to be made as to whether more results are needed.

Signals indicating whether or not a search of the extended index should be conducted

• Number of results – for instance, if there are less than ten results (and that is the signal threshold value)
• Whether the amortized cost of performing the extended search is small, comparing the cost of performing the search to the quality of search results
• Deciding if the user is not satisfied with the standard results returned from the standard index server by looking at something like when a user selects a “next set of results” button repetitively
• When the query scores (frequency and PageRank) of the results are low on average
• if the load on the extended index servers is low
• If for a given query the cost is low (different queries have different costs), or
• Any combination of these signals.

How Estimates of the number of results might be calculated

An estimate might be calculated on the fly, while the search is being performed, based upon how frequently results are being obtained from the standard index servers. For example, the estimate might be based upon a search of a small percentage of the full index – less than 10 percent, and perhaps even less than 2 percent.

Queries sent to the extended server

When the standard index and cache were searched, and there were enough results, as measured by the threshold values for the signals listed above, then results are sent to the searcher.

If not, then the query is sent to an extended mixer, and an extended cache is checked for results. If enough results are received there, those are sent to the extended mixer, and extended search results, with associated snippets, are transmitted to the mixer from the extended mixer.

The mixer would take those results and aggregate them with any standard search results, if there were any. Those would be sent to the query server, and then the searcher.

But, imagine that there weren’t any extended search results located in the extended cache. The search request may be sent to the extended index servers.

Filtering at the extended server

Like in the standard index, there are multiple extended balancers that transmit the search onward to the extended index servers.

Balancer procedures in the extended balancer use a balance filter to perform a lookup operation for each term in the received search query to locate corresponding information in the partition index.

The balancer filter uses the information in the partition index to produce a sub-partition map for each of the terms in the search query.

A map of the extended document index sub-partitions is produced for each term of the search query.

The map can be encoded a few different ways, including as a bit map. The map would contain a bit for each sub-partition of the extended index partition serviced by the extended balancer, with a first value of the bit indicating that the term is found in at least one document in the corresponding sub-partition of the extended index, and a second value of the bit indicating that the term is not found in any document in the corresponding sub-partition of the extended index.

Using combined bit maps

Each term has a bit map made for it. A combined map is made from the bit maps for each term in the query, using Boolean logic matchng what was used in the query itself.

In Google, by default, if you don’t use a boolean operator for your search, the search engine will attempt to perform a search using “AND” for all of the terms (or at least all of the non-stopwords). But, you could use the “OR” operator in your search query, or place a minus sign in front of a term, indicating a “NOT” for that term. The way the bit maps for each term would combine would be based upon your use of those Boolean operators.

This combined map would indicate the document index sub-partitions that may index one or more documents that satisfy the search query, and which document index sub-partitions don’t.

The query is sent only to the extended document index sub-partitions indicated by the combined map as potentially indexing documents combining the search query.

By limiting, or filtering the extended search to only those sub-partitions containing the searched terms, there is a significant reduction of sub-partitions that need to be looked at. This makes extended searches more efficient and faster.

Using sub-sub-partitions

The maps produced could even be based upon sub-sub-partitions of an extended document index partition, instead of sub-partitions. There are fewer documents in the sub-sub-partitions.

A sub-partition might index the terms in approximately a half million documents, and those index sub-partitions are each partitioned into 128 sub-sub-partitions, which means that each sub-sub-partition will therefore index about 4,000 documents.

Returing results from the extended index

If results are found and sent to the extended mixer, snippets are requested from the documents associated with the query terms, and the results and snippets may be saved for future searches in the extended cache, and then sent to the standard mixer.

If no extended search results are found, the extended mixer informs the standard mixer of the lack of results.

If there are extended results, the mixer takes those (with snippets) and aggregates them, with standard search results from the cache or standard index server. Aggregated search results are then sent to the query server, and then to the searcher.

An alternative method for performing an extended search

Similar to the above method, after results have been found in the extended database, the extended mixer determines how many extended search results there are.

The number of extended search results is sent to the mixer, which has already received standard search results and snippets.

The standard results and the number of extended results are sent to the query server, and those are shown to the searcher.

The standard search results and snippets are presented to the searcher, as well as a link stating that the number of extended results can be viewed by selecting the link. That link may be provided without showing the number of extended search results, or before the extended results have even been obtained.

If the searcher selects the link, the search is repeated, but with the extended search results shown to the searcher, providing the user with the standard results, and with results from more uncommon or obscure documents.

A flurry of recently granted patents is starting to cloud the local search space. After being granted a patent for location-based search, Local.com this morning announced the awarding of another patent for an “ad supported 411 local search model.” Local.com’s stock shot up 52 percent on the announcement.

The problem is Jingle Networks has a similar patent. And then there’s the looming Geomas local patent that has both online and mobile implications.

There’s also a lesser-known local search patent that Microsoft owns through its acquisition of Vicinity Corp. in 2002.

How all these potentially conflicting local search patents relate to one another is hard to determine and most likely will have to be resolved through litigation at some point.

I’ve got a patent, you’ve got a patent. It seems that nearly everyone running an Internet company has a patent nowadays. Local.com is the latest to announce a patent in the local space. The patent (no. 7,231,405), which I was unable to locate through a USPTO and Google Patents search, “covers local search technology related to identifying location information from web documents, indexing that information and making it searchable geographically” according to the press release.

Jingle Networks, which operates 1-800-Free411, was recently awarded a locally oriented patent for ad supported directory assistance. And IP holding company Geomas has sued Verizon in U.S. District Court to enforce its extremely broad local search patent.

These patent announcements emerge as something of a marketing (and potential M&A) vehicle for these companies. The whole technology patent area seems to be in some disarray. Given all the patents, one has to expect litigation in order to resolve the confusion and competing claims. And even then we’re probably talking about appeals. The whole process gets extremely expensive. But unless these companies are willing to litigate numerous would-be licensees will likely balk.

The recent U.S. Supreme Court KSR v. Teleflex decision seems to put the courts somewhat at odds with the U.S. Patent Office itself. The court is trying to provide some limits on the enforceability of “obvious” inventions or those that would come in the course of “ordinary innovation.” Postscript: I was sent a link to the patent from Local.com. It appears to have been acquired from a third party.