Common Voice – Mozilla's initiative to help teach machines how real people speak
voice.mozilla.org
voice.mozilla.org
This effort would be more interesting if it could collect speech data in a more specific domain, like web search queries.
How so?
That's literally the worst case scenario. Victims escalating the violence.
Common Voice has optional fields for age and accent in the dataset, but I don't use them. I just toss everything into a single bin.
I've had users with many accents (including e.g. thick UK, India, Germany), report that my 3k hour model including Common Voice performed significantly better for their accent than a librispeech-only model, and also much better than the older macOS speech recognition English models. (And how a model feels for a user in the real world is something you can't really get by just testing against switchboard)
I can't afford switchboard, so I'm very glad that datasets like Common Voice, librispeech, and TED-LIUM exist. I hope at some point I can get the model running in a website, it really does feel pretty good to use.
For Talon we developed a custom speech collection site, that has a lot of prompts we control: https://speech.talonvoice.com - I tried it out with the TIMIT prompt list initially, but right now I'm recording dense command-like speech, which I've found some of my more experienced users are able to say naturally/quickly, not like they're reading from a prompt.
I don't have a separate verification process, because I've had a lot of success with a process that automatically prunes inputs that "obviously make the model much worse". The site is basically designed to record as fast as possible, with keyboard shortcuts to go to the next item and start/stop recording so you can almost record nonstop, which ends up being slightly less forced than just reading one sentence.
The site is pretty reusable, I previously used it at noise.talonvoice.com to record "noise recognition" samples. Here's the source of the current speech site, if someone wants to spin up a common-voice-lite for a specific domain it's pretty easy (just need to run a python app somewhere): https://github.com/talonvoice/noise/tree/speech-dataset
Fair but statistical systems are also extremely susceptible to placebo. I would be curious what a test set shows.
The point isn't that LibriSpeech isn't clean enough. Rather, it's that conversational speech is very different from read speech (which is based on written text). Everything has an effect, even planning of utterances (think: hesitations, "uhm", "uh"), turn taking behaviors (think: how speakers negotiate taking turns), how speakers self-correct, phonetic convergence (think: speakers adapting their speech to be more similar to that of their interlocutor), and so on.
The Common Voice data won't help with that, as it's read speech. It's far more expensive to collect conversational speech datasets, as transcription (or correction of automatic transcripts) involves a lot of manual labor.
The page itself has only very few links. Randomly clicking through it logged in would have a high chance hitting one of the links that would either lead you directly to one of the links where you can CONTRIBUTE: https://voice.mozilla.org/de/speak, your dashboard where you can also find links to the CONTRIBUTE part or just the huge microphone symbol.
The actual method to contribute to them is finding Mozilla's internal translation tool, proposing translations, then finding a Mozillian to approve them. Then there's an obscure webapp for sentence submission and voting, but it doesn't give any feedback whether sentences you've submitted were accepted or went to /dev/null.
(you see people struggling with easiest things, because you assume people struggle with easiest things, rather than that you've misunderstood the problem.)
I've submitted 200 sentences, and none of the progress counters shown to me increased by 200, so I assumed they were lost, and gave up.
Edit:
- There are already paid child actors, you could somewhat-manually collect their speech from e.g. movies and TV shows to have _something_
- Even if there's some copyright issue with distributing their audio directly, it's not clear (uncertain but dubious?) that a model trained on their audio would have any copyright concerns as long as it can't be used to reproduce the original audio
- What is Mozilla going to do if <1% of their dataset is already children who didn't put an age in? Is that a COPPA violation? There's even the defense of "an adult can sound like a child and we also don't know who this child is so how is it personal information" (I have no idea the usefulness of any of that)
I did some of the "listen" exercises to validate how people pronounced some sentences, and I got a few people who spoke with very strong (Indian, Nigerian, UK, ...) accents. How would you take these things into account? Just average all of them and hope for the best? Not sure how to approach that. I don't think it's very straight-forward. Interesting problem though.
However, you can't do anything if you don't even have the data, so props to Mozilla for starting this.
[1] https://ai.googleblog.com/2019/10/exploring-massively-multil...
Also, we would need en-FR, en-ES, en-IT, etc. All languages as spoken from all other native languages. And obviously the strength of the accent varies.
Mozilla DeepSpeech used to release checkpoints trained on Common Voice along with a few others (LibriSpeech etc). But they’ve dropped CV in the latest releases and just rely on the others.
I think the others are more standardised in terms of accents.
It’s likely possible to fine tune a model with different accents — so long as the language is the same then the model can up date the phonemes it recognises.
But accents are definitely a live issue.
There has to be that data somewhere, right?
Nice initiative though.
You can also see a demo here (requires git clone): https://github.com/est31/js-audio-normalizer
1. That the words say what is written?
2. That the words are clear and easy to understand?
3. That the speech is fluent and natural and easy to listen to?
I would guess 1 and maybe 2, and not 3, but it's only a guess because I don't see it written down anywhere.
Many of the clips are very stilted, spoken slowly and unnaturally.(Also, sometimes the text doesn't make sense. "On a normal Hajj, it would be around to walk." But I'm guessing this doesn't matter.)
For those that don't want to read, the only real rule is: All the words are read exactly as written, and no other words are heard.
Clarity, stiltedness, background noise, pronunciation (so long as it's acceptable English), accent, etc don't matter.