I love this idea - can you shed any light on the technologies you are using?
We built an API that processes audio and forms an index for that file. The API search function then goes into that index to look for queries. It doesn't just look for words that match the text you see, but also the way it sounds.