57 karma · joined September 20, 2016
I bootstrapped my personal speech model by retaining the commands from me using WSR. My voice is quite abnormal, and it took only 10 hours of speech data to train a model orders of magnitude more accurate than any generic model I've ever used. And of course, I retain much of my usage now with Kaldi, so my model improves more and more over time. A virtuous flywheel!
I bootstrapped writing it initially using the Dragonfly WSR (windows speech recognition) backend, because that gave me the best accuracy out of the available options at the time. All of my development of it since the initial working version has been done using each previous version, so now it is basically bootstrapped itself. My productivity skyrocketed once I switched to Kaldi, due to being able to use my custom trained speech model just for my voice for orders of magnitude better accuracy, plus dramatically lower latency. (And it freed me from being dependent on closed software out of my control.)
I bootstrapped my personal speech model by retaining the commands from me using WSR. My voice is quite abnormal, and it took only 10 hours of speech data to train a model dramatically more accurate than any generic model I've ever used. And of course, I retain much of my usage now with Kaldi, so my model improves more and more over time. A virtuous flywheel!
Regarding the software packages you referenced: Yes, Dragon is trash that I want nothing to do with, because of its inefficient interface, its complete inability to accurately understand my voice, and its generally shoddy software quality. Voice Computer (which I hadn't seen before) is therefore eliminated as well, though it doesn't look terrible as a front end to Dragon to better use the OS GUI-accessibility info. Many people like Talon, but I demand something open, which I can modify to suit my needs.
Background: I develop kaldi-active-grammar (https://github.com/daanzu/kaldi-active-grammar), a free and open source speech recognition backend usable by Dragonfly, itself entirely by voice. There's also a community of voice coders using Dragonfly and other tools that build on top of it, such as Caster (https://github.com/dictation-toolbox/Caster).
I want to point out that the Dragonfly library has backends other than the closed/commercial Dragon, and there has been significant progress expanding beyond it recently. I have been developing kaldi-active-grammar [0] for the past couple years as a fully open source, modifiable, cross-platform, and free alternative to Dragon when used with Dragonfly. While KaldiAG misses some of the user-friendly niceties of Dragon, these are least helpful when coding by voice. And KaldiAG has benefits such as lower latency that improves coding by voice tremendously.
I created KaldiAG because I didn't trust relying on closed source software for something so crucial to my productivity, where a decision by an outside party determines whether I can function. As a bonus, open source means I can make it work better to fit my needs than closed source ever could.
[0] https://ai.googleblog.com/2019/08/project-euphonias-personal...
However, this means that the gains to be had from personalized training are greater for disordered speech than for "average" speech. I develop kaldi-active-grammar [0], which specializes the Kaldi speech recognition engine for real-time command & control with many complex grammars. I am also working on making it easier to train personalized speech models, and to fine tune generic models with training for an individual. I have posted basic numbers on some small experiments [1]. Such personalized training can be time consuming (depending on how far one wants to take it), but as my parent comment says, disabled people may need to rely more on ASR, which means they have that much more to gain by investing the time for training.
Nevertheless, a Common Voice disordered speech dataset would be quite helpful, both for research, and for pre-training models that can still be personalized with further training. It is good to see (in my sibling comment) that it is being discussed.
[0] https://github.com/daanzu/kaldi-active-grammar
[1] https://github.com/daanzu/kaldi-active-grammar/blob/master/d...
However, kaldi-active-grammar specializes in real time command and control, with advanced features that don't really apply to your use case. Vosk [1] is probably a simpler, better fit for you. It likewise uses Kaldi and can use my models, and offers some others of its own as well.
Neither are particularly focused on transcription per se, but they are open.
The Kaldi engine, being developed primarily for research in speech recognition, can support a huge variety of "models". I think the consensus general best for most use cases (particularly for real time, low latency, streaming use) currently would be considered to be the "nnet3 chain" models, which are what my kaldi-active-grammar uses/supports.
When using commands at my computer, I frequently find myself muttering and grunting things that even I think to myself (that is utterly un-understandable), yet the computer understands just fine. Dictating for prolonged periods can be tiring for me, but I can happily code by voice commands all night.
More info in my other comments here: https://news.ycombinator.com/item?id=23507363 https://news.ycombinator.com/item?id=23507829
And one shameless plug deserves another! Vosk is a great project, but my kaldi-active-grammar [0] (mentioned in another comment here) also uses the same Kaldi engine, but extends it and is designed specifically for this use case. It supports defining many grammars, in any combination, and activating/deactivating them at will instantly per-utterance. I think it's probably a better fit as a backend for your project than vosk. My work focuses on the backend technology, so it would be great to have more front ends using it to put it within users' reach (so to speak).
I develop kaldi-active-grammar [0]. The Kaldi engine is state of the art for command and control. Although I don't have the data and resources for training a model like Microsoft/Nuance/Google, being an open rather than closed system allows me to train models that are far more personalized than the large commercial/generic ones you are used to. For example, see the video of me using it [1], where I can speak in a relaxed manner without having to over enunciate and strain my voice.
Gathering the data for such training does take some time, but the results can be huge [2]. Performing the actual training is currently complicated; I am working on making it portable and more turnkey, but it's not ready yet. However, I am running test training for some people. Contact me if you want me to use you as a guinea pig.
[0] https://github.com/daanzu/kaldi-active-grammar
[1] https://youtu.be/Qk1mGbIJx3s
[2] https://github.com/daanzu/kaldi-active-grammar/blob/master/d...
https://github.com/daanzu/speech-training-recorder
Originally intended for recording data for training speech recognition models [0], it should work just as well for recording to be used for speech synthesis.
Although I mainly use it for computer control as demonstrated in the video, I do have many commands akin to home automation, like adjusting the lights, HVAC, etc.
KaldiAG has an English model available, but other models could be trained. Although you can't just drop in and use a standard Kaldi model with KaldiAG, the modifications required are fairly minimal and don't require any training or modification of its acoustic model. All recognition is performed locally and off line by default, but you can also selectively choose to do some recognition in the cloud, too.
Kaldi generally performs at the state of art. As a hybrid engine, although training can be more complicated, it generally requires far less training data to achieve high accuracy, compared to "end to end" engines.
For what it's worth, my voice is quite abnormal, so most untrained speech recognition is terrible for me, and even performing the normal "training" for Dragon still resulted in very poor accuracy. However, apparently their training is quite limited, because once I developed Kaldi Active Grammar, and did my own direct training, the results were fantastic in comparison, with orders of magnitude better accuracy.
Open source is what allows this.
Also, Dragon has many commands that cannot be disabled, and if your accuracy is already low, more commands available means more possible things to get wrong. Something like Dragonfly with KaldiAG could allow you to reduce the command set, and improve the practical accuracy.
So, more personalized training/software may work better.
I created Kaldi Active Grammar because I didn't trust relying on closed source software for something so crucial to my productivity, where a decision by an outside party determines whether I can function. As a bonus, open source means I can make it work better to fit my needs than closed source ever could.
Furthermore, the original article mentions Caster (which is built on Dragonfly), but doesn't mention that KaldiAG works with it, and that work is underway to expand Caster's platform support.