Initial Release of Mozilla’s Open Source Speech Recognition Model and Voice Data
blog.mozilla.org
blog.mozilla.org
As time passes, the quest for openness and freedom in software moves higher up in the stack. Thanks to the latest ~30 years of effort, we basically came to a point in which we have free OSes, basic infrastructure, building tools, end-user applications.
In the last ~10 years we changed paradigm: autonomous desktop computing progressively transitioned to mobile, with a lot of functionality offloaded to "the cloud".
What I feel is needed, going forward, is working towards building a viable free replacement for these distributed services. DeepSpeech is a step in the right direction.
Edit: just speaking about SW, here. HW is worth a different topic, and probably poses even more challenges.
I hold a rather uncommon view that a path to a more free society requires abundance of open source tools. In the software world we are getting closer, but the whole hardware world is still one big unmoving binary blob.
I make open source robots and I’m hoping to help in that area. But we need open source machine tools, open source factories, and open manufacturing processes. There’s lots of work do be done in hardware land.
I’m happy that we’re moving to more sustainable energy but for me the revolution I’m most interested in is the one where the people control the manufacturing technology. And I hope we’re caring for the earth while we do it.
Oh and see my latest 3D printed robot in that vein: http://reboot.love/t/rover-a-robot-you-can-make-at-home/94
Well, the human brain manages to master several complex tasks using much smaller data sets than current machine learning algorithms. Natural language acquisition, for example, seems to require fewer than 10 million spoken words per year, and even academically successful 12-year-olds might be reading 1 to 4 million words per year. These are not exactly tiny data sets, but they don't require Google's scale to recreate, either.
Sure, the human genome probably "knows" what set of models to try when building the brain, which gives it an advantage. But I can't think of any reason why machine learning couldn't ultimately try similar techniques with similarly-sized data sets, and get competitive results.
I didn't phrase it correctly. What is algorithm independent is the theoretical performance bound for a given data complexity.
Sure, not all algorithms are the same, but the best performance that can theoretically be achieved with a given iid sample of a given size is algorithm independent.
So yes, a better algo can get same performance with less data, but limit stays the same.
Technically speaking, all of these 'voice assistants' and the like would be far better products if they hosted a local pre-trained network that did all of the recognition. Latency is the biggest challenge to these systems, and that simply will never be solved so long as the recognition happens on a server miles away from where the user is speaking. The speed of light, at a minimum, comes into play.
What would be very interesting would be if there was a development of a continuously learning system which actually performs better training itself to recognize one or a few users voices without the burden of carrying the weights necessary to also recognize the voices of people with accents on the other side of the planet from anyone who will ever be within earshot of it. That would be an even more overt and active disincentive to sending all your data away "for speech recognition" (the spying is a fringe benefit!) and sitting around waiting for the cloud to get back to you.... and I imagine we'd see it ignored.
The limitations today aren't technical, they're organizational and business oriented. And those things don't generally change in concert with technical changes.
I would love an open-hardware microphone array that I could use with a Pi or something similar, to write my own Alexa. Not only would I love this, I would store all my commands and send them to Mozilla to help with their speech recognition models.
I don't want to be the guy who wishes someone else would do all the work and then give it away for free, so I'll do what I can to help (which is probably limited to writing a bunch of code and documentation on how people can set this up more easily). Congratulations and thanks to Mozilla for this.
I’d love to see one of these meshed together with this new Mozilla voice project for an open source Echo or Google Home. The only missing piece it seems at this point is NLP and all of the glue that converts commands to API calls.
Both this data set and LibriSpeech are read speech, where the speaker was prompted with a transcription and asked to say it out loud. In practice it's very rare that you're trying to transcribe speech that's already been transcribed. Speech patterns for computer-directed speech (e.g. for voice activated user interfaces) or human-to-human speech (e.g. for meeting transcription) are quite different.
That said, we will definitely be exploring ways to build in organic speech and perhaps transcriptions to the Common Voice app. This will solve another problem for us too, which is getting public domain material for people to read. Doing this obviously requires a much more complex user experience, and we have more work to figure out how to make something that people will want to use and contribute to. Stay tuned for that :)
On the flip side, we hope that these datasets, models, and the tools (ie. DeepSpeech) can get more people (researchers, start-ups, hobbyist) over the hump of building an MVP of something useful in voice. Once you have people using your products, collecting useful in-context voice data becomes much easier.
On that note, another approach we are working on is partnering with universities and socially-aware startups like MyCroft, SNIPS, and Mythic. Imagine if voice products in market allowed their users to opt-in to contributing their utterances to an open resource similar to Common Voice. Of course, sharing your voice publicly is not for everyone, or every product scenario. But it does work for some. And if we pool our resources, our hope is to indeed commoditize speech-to-text so that we can focus on more interesting challenges like building voice experiences people want to use. (For instance, could voice somehow be a "progressive enhancement" to the web?).
Though, most videos of decent length would only contain say three or four speakers, which is most definitely sub-optimal.
https://www.youtube.com/results?sp=EgYYAigBMAE%253D&search_q...
I have created my own TamperMonkey plugin that adds TTS to web pages. It finds text, makes it clickable, and when a user clicks a word, it starts reading from there, highlighting text as it reads it, skipping menus and chrome. I find this helps me better focus on reading. Unfortunately I can only stand one single voice and it's been stagnating for years (Alex from Mac OS). Can't wait to hear the WaveNet voice Google has been threatening to give us.
The bigger problem is, of course, that you need speech data with (fairly) accurate transcripts for training ASR systems. These typically don't exist for call center calls.
I really wish Mozilla would release a keyboard app for Android. It would instantly be the single most trusted keyboard available.
http://cmusphinx.sourceforge.net/
http://julius.osdn.jp/en_index.php
Maybe with this data set released eventually all that additional data will be used to improve those tools as well
My personal problem with Kaldi is that I don't have enough RAM in my cheap laptop to work with any of the big models. When it started swapping just doing the preprocessing for one of the pretrained models [2], I kind of abandoned that project until I bother to get new hardware.
For that reason, I can't tell how good the pretrained models really are.
[1] http://kaldi-asr.org/doc/kaldi_for_dummies.html [2] http://kaldi-asr.org/models.html
It’s a great idea to crowd source this. Wonder if this project can turn voice recognition into a solved problem.
I just set a daily reminder so I can do 10 minutes a day.
I also want to emphasize the importance of listening (validating) as well as recording. Validation is an big part of the puzzle for building machine learning viable data.
And should we validate whether they match the proper accents as well? e.g. if I hear a clear Dutch accent, I presume you wouldn't want that labelled "native British speaker"?
Well done, people!
As voice recognition becomes more sophisticated I think captchas are going to have to evolve to kjeep up as well.
So today, failure to captcha is actually an indication that the other end is human.
Coyld this be used to train a model/engine that can be used that way?
Edit: NM found it under releases: https://github.com/mozilla/DeepSpeech/releases.
I'd have to assume DeepSpeech outperforms anything running on a RasPi3, at least for LVCSR. It hits 93.5% accuracy on Librispeech, which I've never seen from any offline recognition models.
[1] https://github.com/kaldi-asr/kaldi/blob/master/egs/librispee...
[2] https://groups.google.com/d/msg/kaldi-help/Pr6jPH1Qshg/kn8df...
I think their product is higher level - it has things like recognizing commands and sentence structures, and hotword detection.
While i do believe that's interesting, I don't understand why that would be Mozilla's job.