Google Translate, Now With Voice Input

http://chrome.blogspot.com/2011/03/talking-to-your-computer-...

SR is roughly divided into acoustic and language modeling. The acoustic model proposes words that might have been said given some chunk of speech and the language model tells you what the most likely actual word is given what's been said.

The acoustic model can be solved in a large number of ways — though production technologies use very large hidden Markov models — but decoding a word sequence from speech might scale like O(knm^2) with n being the size of your vocabulary (often large), m the complexity of the acoustic model (# of phonemes modelled, perhaps) and k the number of acoustic frames. The n at the very least can be parallelized (embarrassingly), but the m and k cannot.

The language model involves a search through an exponential search space of orderings of words in the vocabulary (n^l choices, but l is also unknown). Anything sophisticated also will have an incredibly large (in memory) model as it has to have parameters across words, pairs of words, triples of words, grammatical categories, topics, etc. etc.

Solving both of these problems well simultaneously is not a task for a consumer computer. Speedy algorithms with small vocabularies and simple models exist and are implemented (Dragon Naturally Speaking, for instance) but Google didn't go and record a million hours of GOOG-411 to produce Dragon's technology over again.

---

Finally, there's a lot of work on front end signal processing in SR. Before you get into acoustic and language modeling, you often transform your input into another representation (often spectral components from sliding 10ms frames). A growing camp in SR research involves finding sparse front end representations of speech though. If the client-side software is capable of quickly computing a sparse representation of the speech, that could dramatically reduce the latency and bandwidth issues.

In order to do the kind of open domain recognition google (and others) do here, you need data. Lots of data. That implies that the models themselves are very large, and (generally) require more CPU to recognize with. You can't do that on a mobile device, mainly because of the space issue, but fancy new algorithms probably consume more CPU than even the latest mobile hw can manage[1]

The second, related issue is that, again, in order to do this kind of open domain recognition, you need to constantly improve the models[2]. Even a moderately sized set of models would be a pain in the rear to send back up to every android device using the system.

[1] You can typically do the front-end feature processing (and adaptation) on the device, and most vendor's solutions end up doing a bit of that.

[2] Even for speaker dependent modelling (dictation), you'd probably want to associate a key with the model and adapt that model in the cloud rather than sending model updates back to the device because of the space and transmit times.

Not client-side, or I would use it more often (lot of dead spots near my house due to the peculiar geography).