I'm curious but a bit afraid to test it out.
The idea of having a model of my voice out there that can say whatever is written in a text box is scary.
The idea of having a model of my voice out there that can say whatever is written in a text box is scary.
The way it works is, a model is trained of all possible voices. Then your specific voice is projected into latent space.
That's why it can mimic your voice with only a few seconds of audio. It's not making a model, but rather using an existing model.
It may seem like a pedantic distinction, but it's why the model isn't as worrisome as it seems. It can't target you specifically, just the average voice near yours.
It's closer to a really talented parrot than a model that can impersonate you on command. I suspect if you try it out, you'll be surprised it's so far off from your actual voice.
The English->Portuguese sample sounded nothing like me at all except for one syllable where it sounded like it was playing back a brief snip of what I had recorded.
The English->French version did a little bit better, it sounded like the voice had been influenced by mine in some small way.
English->English (saying a very different sentence to what I recorded) was pretty impressive though.
At least there, it was optional.
Perhaps a solution is a sound fingerprint requirement for voice imitation software so that it's easily identifiable in court if it's an imitation voice.
It's somewhat of a new frontier, imagine during a divorce proceeding your ex-partner fabricates voice recordings of you threatening the kids so you don't get custody, how do you protect yourself against that, how to you prove that's what happened? Soon enough it'll just be an app on their phone that they use to record your voice during a discussion, then later spits out a sound file of you saying whatever they want you to say. That's clearly a socially dangerous tool.
Fake calls to your relatives in your voice or even fake video with your face and voice asking for money! or illegal activities.
Few years later a company will come and say we can detect if it's fake or not pay $10,000 for solution, or get ready to be in prison. Oops! legal system doesn't accept this as a proof, now what? Welcome to the prison.
Both companies are making money, and you are paying by money and your life.
I can see the government banning using voice as a password. I can't see it banning the tech. The criminals will use the tech regardless of if it's banned. Looks like we'll need person to person authentication for our relatives soon.
If technology like this is plausible, then the recording shouldn't be considered a statement by me in the first place.
People are just going to have to learn not to trust audio. People adapted to photoshop, they'll adapt to this.
It's a valid concern to not want to give a random website a workable voice model. Just because you've talked on the phone or used speech-to-text before, doesn't make that concern invalid.
Probably the folks who could best use this nefariously are the folks we already know, who have much greater availability to our voice. Those folks are in the best situation to capitalize on a working voice model to, say, call our manager, bank branch, or local emergency operator. A random website would have to go to some effort to accumulate the needed information to use our voice for much, whereas someone who already knows us could have us fired for the contents of a phone call to the manager, up on charges for prank-calling 911, or worse.