This is not accurate. At least on the Google home the "wake word" is baked into the hardware and no audio gets through that piece of hardware until the wake word is detected.
Obviously there can be defects, but Google is not getting 100% of the audio from your home.