I'd be curious to see the WER on out of vocabulary words, seems like this cannot generalize to unseen word's given this step: 'The closest of ~229,000 English words wins, and the slices become a transcript'
You can just embed those new words and they get added to vocabulary. It is fairly simple. One risk is - When a new word is very close to a known one, it tends to fall to it.
Would it make sense to rather compare distances to sub words rather instead full words ?
I don't know if this would be possible because now you have to deal with sequence modeling to transform audio to sub words to words to sentences.