Besides, when they can hack the database itself and get a list of what they know to be passwords, why wouldn't they just do that instead of hacking a bunch of voice snippets and combing through them hoping to maybe find where somebody said a password. It's stupid.
People can use the service and come to terms with that fact or they can not use it. If they choose not to use it, they need to live with the consequences of that decision. If it means they have no other way to access some service, then that's something they need to take into account. I can't think of any service in existence that can only be accessed by voice recognition instead of having an phone representative, physical branch, or other options. If you really, really need to access your bank account, there are plenty of options aside from speaking to your smart TV. Nobody is getting marginalized here.
Or you just pass the data to Siri or Cortana (or whatever microsoft is calling it). Protecting against hacking is defense in depth. If the database is well protected and monitored, attack the target that is not well protected and monitored.