To the other point, I'm not entirely sure I'm comfortable with the patents, but it's at least an Echo simply gleaning more from a voluntary engagement with it. I think the general feeling is an Echo is "spying" when it engages itself to listen to your home.
Of course, I have no way to know if my echo is doing that, or if it were to start, and it's out of my hands to prevent it. The same with a smartphone.
If Amazon wanted to spy, probably a smart tactic would be to deliberately "flaw" the wake word recognition such that it could plausibly be a bug, but the result is activating very often randomly. They already kind of do that, it would just need to be more frequent.
In any case having the capability to spy on you is fundamentally different from having the actual on-paper intent of doing so, such as a patent.