Do you allow computers and cell phones? Or is there something special about a microphone when it's attached to something shaped like a hockey puck instead of a box?
I physically remove the microphones and webcams in my laptops and use a USB microphone with a physical power switch. This is probably unnecessary as, by the same logic as my phone, nobody should have access to these peripherals unless I grant it. But it helps with paranoia. I don't like a camera and microphone pointed at my face constantly.
No way in hell would I buy a device where "always listening" is listed as a feature.
In both cases, you have to trust the manufacturer that the device behaves the way they say they do.
I say this not to belittle you or to minimize your feelings or experience, but you have to understand that, among the general population, your needs/wants here are in a very tiny minority. The number of people who do what you do is vanishingly small. The vast majority of people do not care, and those who do will often make a conscious decision that the trade-off is worth it. And that's ok! As long as you can continue to take care of your needs, and other people can take care of theirs, then all is well.
I find it laughable that folks may try to steal things from you with software once they have physically breached your house. At that point, they could just steal your wallet, your car, your Echo etc.