Have a look at the openWakeWord model which is especially built for detecting wakewords in a stream of speech.