Meta says its new speech-generating AI model is too dangerous for public release | Hacker News Reader

Meta says its new speech-generating AI model is too dangerous for public release

theverge.com

16 pointsludovicianul17 comments

We already have voice cloning with Eleven Labs and at least one other good one.

We are getting close to the point where you search for "clone person" and ten different websites come up that will make a video of your subject saying or doing whatever you ask for less than $200. You can already do some weak versions of this. Will probably be very good within a couple of years.

I assume its already illegal to use this to impersonate people. But that doesn't stop the technology from becoming popular. People will just add a checkbox to the page that says "I certify that I have permission..".

Within five years it will be so easy to do that people might not even take impersonation seriously anymore. We might need to start relying on some kind of cryptographic identifying signatures or something.

Oh.. I forgot.. we may almost be there already. https://m.twitch.tv/trumporbiden2024/home

Actually you could imagine say in five years or so the technology is so good that people use it to "be" in multiple places at once. They train an LLM with all of their work and ideas, another model that can simulate their face, speech generator, and hire it out.

Imagine CBS in 2033 decides that (since it's now an affiliate of Tencent) that they want to do a Chinese version of the late show actually featuring Stephen Colbert. Probably not even particularly hard at that point since it doesn't need to be live.

> We might need to start relying on some kind of cryptographic identifying signatures or something.

Maybe.

More likely, the already dangerous level of distrust in our (US, anyway) society for pretty much everything will just deepen and become more entrenched.

Voice cloning is already a thing so this seems like a PR attempt to drum up noise frankly. "It's so powerful and dangerous, you can't have it but I have to tell you about it anyway".

May be an effort to not add oil to the fire

This is exactly my take. If I were sitting on a technology that I thought was too dangerous to release, the very last thing I'd do is tell anyone about it.

This is just a transparent, and slightly pathetic, PR move.

What a bunch of baloney. By this rationale electricity is too dangerous, the internet & browsers for that matter as well. Shhhh lest someone compile their own operating system.....

Compared to OpenAI's antics I think this is actually a very reasonable move, and a good example of the less glamorous but more concrete reality of actual AI risk management. There's a concrete harm -- disruptive ease of impersonation over common voice communication channels. Tools for this will diffuse out anyway, because at the heart of the problem is a bad cryptography-style security assumption. We assumed voices were expensive to clone in real time with high fidelity, turns out maybe they aren't. In time, societal expectations will shift, like they did with photographs, but it sounds like Meta is making the responsible choice to not accelerate the cats ahead of the mice.

I still think it's marketing because if it really is dangerous, why talk about it in the first place? (I don't doubt that it may be 'dangerous' if used for nefarious purposes, but if I have something dangerous, I wouldn't go announcing it to the whole wide world).

It’s absolutely marketing.

“This heroin is too strong, just had 2 clients OD last week, you sure you can handle this?”

Just wanted to say that I loved the analogy :)

Matter of time & effort since the arch is out but without much details of the layer blocks just number of params, heads etc.

I honestly don't think we would have the internet or electricity if it was a new thing today.

It would be impossible to run power lines. We would make some argument that we need to slow down and stick with candles for now. We already have enough to worry about with fires. You want to run high powered electric death lines into people's homes and water? What are you crazy or you just don't understand what happens when water and electricity mix?

The cat is out of the bag: https://youtu.be/ymZgpH7l1_k

I don’t think we can stop what is about to happen.

We will only learn by experience whether the ai-doomers were right.

As of now, this video seems to be made private. Could you please share what was in it?

Let's wait for another leak then.

How is this different than photoshop for making fake images.

Fake video and fake audio just seem like more of the same.

Just like we haven't been able to trust photos for a long time now, we just need to add video and audio to that.

Way to go.

Reply on news.ycombinator.com