Shameless plug, but I have been working on an open source IDE plugin [1] for the IntelliJ Platform which attempts to do this. Previously, we used an older HMM-based speech toolkit called CMUSphinx [2], but are currently transitioning to a deep speech recognition system. We also tried a number of cloud APIs including Amazon Lex and Google Cloud Speech, but they were too slow -- offline STT is really important for low latency UX applications. For navigation and voice typing, we need something customizable and fairly responsive. Custom grammars would be nice for various contexts and programming languages.

There are a few good OSS offline deep speech libraries including Mozilla DeepSpeech [3], but their resource footprint is too high. We settled on the currently less mature vosk [4], which is based on Kaldi [5] (a more popular deep speech pipeline), and includes a number of low-footprint, pretrained language models for real-time streaming inference. Research has shown how to deploy efficient deep speech models on CPUs [6], so we're hoping those gains will translate to faster performance on commodity laptops soon. You can follow this issue [7] for updates on our progress. Contributions are welcome!

[1]: https://github.com/OpenASR/idear/

[2]: https://cmusphinx.github.io/

[3]: https://github.com/mozilla/DeepSpeech

[4]: https://github.com/alphacep/vosk-api

[5]: https://github.com/kaldi-asr/kaldi

[6]: https://ai.facebook.com/blog/a-highly-efficient-real-time-te...

[7]: https://github.com/OpenASR/idear/issues/52