This is my concern as well.
My privacy threshold is pretty low (my username across most of the web is my real name, for example). This, though, is an always-listening device in your home, that connects directly to a company who's stated goal is to sell you more things. That, combined with the potential for abuse/hacks/subpoenas, crosses a line for me.
It feels a lot like the GPS trackers that insurance companies want to put in your car, which I also think is a bridge too far.