Hacker News Reader

Comment by bmn__ | Hacker News Reader

I do not understand how to use Deepspeech even in the most simple use case.

1. I want to teach it ten words. How do I do this?

2. I want to speak into my microphone (available as a Pulseaudio device) and recognise the words and output the words as a text stream on stdout. How do I do this?

This is the documentation:

https://deepspeech.readthedocs.io/en/v0.6.0/Python-Examples.... https://deepspeech.readthedocs.io/en/v0.6.0/Python-API.html

It does not answer the questions I have.

https://deepspeech.readthedocs.io/en/v0.6.0/DeepSpeech.html

The introduction page is full of incomprehensible jargon.

View on HN

So, funny story: I wanted #2 and did it myself. I was similarly frustrated with the lack of documentation.

It's been a little while since I got it running, but I basically got a siri clone working. If you want to test it out, I can try to answer questions / whatever problems pop up.

The code is here: https://github.com/shawwn/DeepSpeech/commit/01f5cf8d39c356ae...

As far as I know, you can simply run speech_to_text.sh. It will connect to your microphone and start dumping out transcribed audio to stdout.

It wasn't super easy, but once you spend a little time with the code you can sort of figure out ways to get it to do what you want.

EDIT: By the way, people will try to convince you that you're nuts and that the documentation is crystal clear and so on. Know this: It's not just you. I had the exact same experience. It's a recurring theme in AI programming.

The only thing to do is to either find someone else who feels similarly, or roll up your sleeves and dive into the code.

>By the way, people will try to convince you that you're nuts and that the documentation is crystal clear and so on

No it really is simple: the original implementation had a base of prefabulated amulition grammar, this was surmounted by a malleable logarithmic function in such a way that the two main spurving vocabularies were in a direct line with the panametric grammar fields. The latter simply consists of marzlevanes fitted to the ambifacient morpheme waneshaft to eliminate side utterances. This main winding is of the normal lotus-o-deltoid type, just placed in panendermic semi-boloid stators. Basically every second conductor is connected by a nonreversible tremmie pipe to the differential grammeters.

All these words are in: https://en.wikipedia.org/wiki/Turboencabulator

> 1. I want to teach it ten words. How do I do this?

I'm assuming you mean "my target vocabulary is only ten words". In that case, you have a couple of options:

1. Just use it as is and see how it does.

2. Create a language model containing just those 10 words so that the engine is constrained to that vocabulary, which should improve accuracy.

3. Train a (smaller) model from scratch, which requires (audio, transcript) pairs. This is basically research work, there's no recipe you can just follow. The advantage is that if you can make it work, it can be a much smaller model than a model trained for general English dictation.

> 2. I want to speak into my microphone (available as a Pulseaudio device) and recognise the words and output the words as a text stream on stdout. How do I do this?

We have a few examples with microphone input, but they're all based on voice activity detection: https://github.com/mozilla/DeepSpeech/tree/master/examples

I should write an example that is just continuous streaming and output without any voice activity detection.

I'd be happy to discuss further on Discourse: https://discourse.mozilla.org/c/deep-speech

>Just use it as is and see how it does.

How? The docs seem to be suffering from the same problem as the git docs: they assume the reader is already a domain expert, which would seem to run counter to the stated goal of being simple (and ubiquitous).

I tried to figure out how to "just use" deepspeech and failed.

There's a basic example here transcribing from an audio file: https://github.com/mozilla/DeepSpeech/tree/v0.6.0/#project-d...

There are also examples transcribing from the microphone as I mentioned above: https://github.com/mozilla/DeepSpeech/tree/master/examples/

I also have a small GUI example here [0], click once to start recording, once again to stop and show transcript. It receives the same command line arguments as the ones shown in the README link above (namely --model, --lm and --trie).

The README starts with a very basic installation and usage guide, I don't see how that's assuming any expertise.

[0] https://github.com/reuben/ds-qttest

> There are also examples

Fyi, the other 4 example links there are 404. Also in the root README. The linked README's are missing, or diversely suffixed and mislinked.

You are right, and I expect Reuben and team will fix this and are grateful for the heads up.

I was able to find the right links though, for example the WAV transcriber is at : https://github.com/mozilla/DeepSpeech/blob/v0.6.0/examples/v...

(Nod. I may have edited that out from under you - sorry. And thanks.)

Ugh, sorry. I fixed it on master. We'll probably cut a v0.6.1 soon to publish small fixes like these.

Thanks for all your work - it's appreciated.

I think you're being a little harsh.

1. Train it using the training dataset but change all word labels that aren't in your 10 word list to "OTHER_WORD" or whatever. There's probably not much point doing this. Alternatively I guess you can restrict the beam search to only look for your 10 words. Yes this is really complicated - it's a speech recognition engine - they are really complicated!

2. The API docs look pretty clear to me:

> createStream()[source]

> Create a new streaming inference state. The streaming state returned by this function can then be passed to feedAudioContent() and finishStream().

> feedAudioContent(args, kwargs)[source]

> Feed audio samples to an ongoing streaming inference.

> intermediateDecode(args, kwargs)[source]

> Compute the intermediate decoding of an ongoing streaming inference. This is an expensive process as the decoder implementation isn’t currently capable of streaming, so it always starts from the beginning of the audio.

Huh well that's a flaw the neglected to mention! I guess you can't really do what you want yet.

> Huh well that's a flaw the neglected to mention! I guess you can't really do what you want yet.

The documentation is outdated. That is no longer true, intermediateDecode is cheap. Thanks for noticing, I'll fix it.

Reply on news.ycombinator.com