It appears that Google is a couple years away from this coming but they would build on their existing Google Translate and Google Voice features to create an instant translator. You would speak to it in any language, Google would then convert that and spit it back in the language of choice. Google Voice, Google Mobile search and similar products already take use of speech to text services. Google Translate already has some languages that speak to you. Bridging the two together and improving on the underlying technology, can make these feat possible.

Franz Och, Google’s head of translation services, told The Times Online:

We think speech-to-speech translation should be possible and work reasonably well in a few years’ time.

Clearly, for it to work smoothly, you need a combination of high-accuracy machine translation and high-accuracy voice recognition, and that’s what we’re working on.

If you look at the progress in machine translation and corresponding advances in voice recognition, there has been huge progress recently.

Postscript From Danny Sullivan: Google actually demonstrated this software live and working in December. See our Liveblogging The Google’s Web Search “Evolution” Event’ post from then.

Google now says it understands a range of English accents, and Mandarin although it doesn’t yet get all accents in Mandarin. In addition, the capability will be coming soon to the Android and iPhone platforms in China. Dell has introduced a yet-to-be released Android handset (Mini 3i) and the iPhone just launched with the number two Chinese mobile carrier China Unicom. According to the Google Blog:

Although this only works on the Nokia S60 at the moment, we’re working on adding support for Mandarin speech recognition to our products on other mobile platforms, such as Android and iPhone. And bear in mind that this is a first version of our system in Mandarin, and it might not be as polished as our English version. For example, if you have a strong southern Chinese accent, it might not work as well as for people with a Beijing accent…

There are almost 700 million mobile users in China, more than 2X the US population as a whole. China Unicom reportedly has roughly 140 million subscribers. The largest US carrier Verizon has 89 million mobile subscribers. China Mobile, the largest carrier in China, has roughly 500 million users.

Here’s Google’s attempt at vintage “news on the march”-style newsreel footage. Unfortunately there’s no demo of the Mandarin capability:

Click here to view the embedded video.

Google has created a nice chart showing the difference in features between using a Google Voice number or using your own number with Google Voice:

If you decide to use your existing number, make sure to read the FAQs and help documents carefully. Here is a video showing the various features you get with using your existing phone number:

The report, which can be downloaded from the ChaCha site, was sponsored by ChaCha but the authors said, “While ChaCha sponsored the study, it was fully removed from the planning, testing, analysis, and reporting of this work.” The methodology was very specific and relatively elaborate. I summarize it below.

Both Google Voice Search and Yahoo oneSearch with Voice (powered by Vlingo) use automation for voice search. ChaCha uses humans to interpret voice queries, which are submitted through an 800 number: 1-800-2-ChaCha. Queries can also be submitted via text or online.

Here are the bottom line findings from the report:

Source: Albright Communications/MSearchGroove

Here’s how the two categories were defined:

• Natural language queries: queries submitted as questions using normal conversational syntax
• Keyword queries: queries seeking the same results as the natural language queries but reduced to the least number of keywords . . . necessary

What the table and findings above show is that in a “natural language” context, ChaCha was substantially better than Google and somewhat better than Yahoo/Vlingo. In a keyword query context the results are somewhat closer, but ChaCha still comes out on top.

One might expect this result given the involvement of human beings at ChaCha vs. pure automation with the others. From a pure user experience perspective, putting aside the back end, the report finds that ChaCha offers greater accuracy and reliability than either search engine when it comes to voice input.

The methodology:

All searches were performed by a single individual on the iPhone 3G. Here’s an abridged but otherwise verbatim description of the methodology used by the report’s authors:

To assess overall performance of the voice-enabled search services in a typical range of use cases and scenarios, we created 18 queries representative of mobile search usage and trends.

The queries covered search categories considered common in the mobile environment, such as navigation, directions, local information, general information on timely topics, and specialized or unusual long- tail topics. We included specific queries that represent the most popular mobile search terms in 2008, based on mobile search data publicly reported by AOL and Yahoo, as well as social queries. We did not include transactional queries . . . associated with downloading music and applications . . . and purchasing goods and services via a device.

Microsoft’s Tellme and Nuance Voice Control were not included in the study.

There are other improvements, too:

• Transit and walking directions are available for more than 250 cities
• Business listings have been upgraded to include store hours, prices, and ratings/reviews
• Google Latitude had a location detection bug that’s been fixed
• A new “Updates” feature in Google Latitude makes it easier to communicate with friends

This update isn’t an automatic download; Google Maps for Android users will have to go get it via the Android Market.

First let me show you Yahoo’s voice search for a test on [who is barry schwartz]:

Step 1: Click on the “Press + Speak” button to activate voice search:

Step 2: Speak when button turns red and then tap it again when you are done talking:

Step 3: The button turns blue to show it is processing your search:

Step 4: Yahoo shows the search result, and in this case, wrong (I tried three times):

Google’s voice search works a bit differently.

Step 1: Either put the phone to your ear or click the microphone icon:

Step 2: Just speak and when you are done speaking, Google knows and shows that it is processing your search:

Step 3: Google displays search results:

So, Google is one step less, which is nice for a mobile search app.

The My Location feature lets you search without needing to input a city name or some other geographic modifier. Voice search is what it sounds like — a way to search by speaking your query.

Google had offered voice search as a separate download for its Google Maps Blackberry app last summer. They launched the Google Mobile App for Blackberry this past September.

Before showing you that, let me link you to some of the coverage we have seen so far. Matt Cutts of Google has not only one, but two useful posts demonstrating the features. Waxy shows the behind the scenes on how it works. Plus you can see more coverage at Techmeme.

Without further ado, here is my video, demonstrating how it runs on my iPhone. Hope you like it:

And that screen capture I promised you in the video:

Unfortunately, no one has had an opportunity to try the updated app yet. But, if it works as promised, it should be quite impressive.

The technology behind the new voice search capability is built partly on the same platform as Goog411, but apparently that’s only part of the story; there’s a good deal more going on, as well. In fact, Goog411 is reportedly improving and benefiting from the work done on voice search for the iPhone.

The first thing that is both intriguing and very different is that there are no buttons to push to initiate voice search. Once the app is open on the iPhone you hold it up to the side of your head as though you were going to talk on the phone and simply speak the query. Search results then appear as they would if you had manually entered a query.

Google says it has learned tremendously from its experience with Goog411 but its desktop search query data is also contributing knowledge to the effort. These and other technical factors beyond the scope of my expertise will make the system more accurate than what has been possible in the past, said Google’s Cohen.

Beyond its reported accuracy, the usability of the system is striking. Most voice control on mobile handsets requires that buttons be pushed. There’s also often a “walkie-talkie”-style experience, with the phone held out in front of the user to speak the query or command into the phone.

By removing the need to push a button and simply mimicking the experience and handset position of talking on the phone, Google’s voice search may prove to be quite a bit more natural and intuitive. Another benefit for Google is that by having the phone’s receiver closer to the mouth of the person speaking, the system gets a better, cleaner input.

I said yesterday in my postscript to Matt’s post that none of the voice apps or voice-initiated search tools currently in the market have proved to be a kind of “killer app” for mobile. That’s largely because of uneven accuracy and success rates or some other limitation or awkwardness. After my discussion with Google yesterday, I became hopeful that what Google was going to introduce would be a leap forward.

If it is, we should see increased query volumes and longer query strings — and increased search monetization from mobile for Google. But all this will become more clear once the app launches, hopefully today.

Voice search is launching in US English and will roll out to other handsets — and eventually other countries and languages — in the future.

With the new Google iPhone app, you’ll be able to speak your search query into the phone rather than having to type it out. The Times article explains what happens after you ask your question:

“The sound is converted to a digital file and sent to Google’s servers, which try to determine the words spoken and pass them along to the Google search engine.

The search results, which may be displayed in just seconds on a fast wireless network, will at times include local information, taking advantage of iPhone features that let it determine its location.”

The Times article says Google’s updated iPhone app should be available Friday. It’s a free update, and Google expects to make the voice search feature available on other phones, too.

Postscript from Greg: I just checked and the app doesn’t yet appear to be available in the iTunes store. As the Markoff article mentions in the Times, both Yahoo and Microsoft have a variety of voice search apps.

Microsoft’s Live Search mobile client app has offered voice for some time, but it’s limited to mostly local categories. Microsoft also owns Tellme, which offers both the free directory assistance alternative 800-555-Tell and an app for BlackBerry that is voice powered. Microsoft also separately operates 1-800-Call-411.

In July Google introduced a voice-powered version of Google Maps for BlackBerry. Earlier this year Yahoo introduced voice for oneSearch (full web search) with Vlingo. This is what Google is now introducing essentially.

As mentioned, there are also a host of free directory assistance alternatives in the market (six or seven major ones), including Tellme, Call-411 and Goog411. The recognition engine behind Goog411 powers the new iPhone voice search.

Among the range of “voice search” and free directory assistance alternatives services now in the market, mobile ChaCha is noteworthy because it allows users to ask any question, essentially full web search, not just “what city what listing.” It also uses humans behind the scenes to disambiguate and answer queries. ChaCha has said that some of its heaviest users are doing query volumes that mirror search activity levels on the desktop. Hypothetically the more queries, the more ads — potentially.

In terms of the iPhone, a range of companies are working voice “search” or control for the device, including Nuance which previously demo’d voice control and search for the iPhone. The first to officially offer a voice capability for search was Dial Directions, which has two voice controlled iPhone apps, Say Who and Say Where. The former is voice control for contacts/dialing and the latter, introduced several months ago, is voice-based local search through selected sites via the Safari browser.

The key with all of these voice applications is the accuracy of the speech recognizers. The failure rates remain an issue, although they’re improving and variable from app to app.

So far voice has not proven to be the “killer app” for mobile search (as I once believed it would be). However Google’s new voice search iPhone capability should be helpful in selected situations — while in the car, for example, when directory assistance is most heavily used or for longer “long tail” queries that are highly specific. Indeed, it may result in longer or more precise query strings.

Regardless, I’m sure the new Google voice search capability will be quite popular among iPhone users.