If the voice interface is implemented locally then it's not a problem. Google already has it working on the Pixel. It has some limitations but it responds faster and works offline.
https://techcrunch.com/2019/03/12/googles-new-voice-recognit...
https://techcrunch.com/2019/03/12/googles-new-voice-recognit...
That's not really observable or enforceable by me though. A question I ask myself all the time before I say yes to some kind of "new data tracking" is: "How do I think company_x will behave in 10 years if their profitability is plummeting?" Because I think that paints the worst case for companies that so far have been good actors.
I guess the TLDR is: That's great, but I think that the solution is legal protections for users. Right now all the power is in the hands of the collecting organization.
Voice interfaces today work like keyloggers by design, and we're just trusting them not to abuse all the data they're getting. It'd be a lot better if we didn't have to.
(I'm on board with legal protections too, of course.)