Hacker News Reader

Comment by madeofpalk | Hacker News Reader

Uh, I mean, I think we've seen first hand that generally one aspect of safety we want is to not produce sexualised images of children.

View on HN

Is it even an attainable goal? It seems an NN with less than say 4 billion parameters will be able to do that. The cost of training will likely go down with more models being available. Unless we lock down the computing for majority of people, I don't see how we can prevent someone creating a CSAM model in their garage.

I don't want to see CSAM created, but the totalitarian control required is too much for my taste (and frankly it's preferable for that person to use NN than to go out and hurt actual children).

Not to mention even locked down technology is often being abused by the privileged.

People can draw naked children with a stick and some dirt. I’m not sure that preventing the creation of fictional csam is the best use of our resources if we want to protect minors from abuse.

The best use? Probably not. But if I built a website that let people generate extremely convincing unlimited photos of you wearing an SS uniform and forcing your dog to smoke meth and sent them to everyone you’ve ever met, this might seem like a less worthy hill to die on. Or is that just a sticks and dirt thing too?

Oddly specific... Did that really actually happen to you?

Everybody I care about would know that those pictures are not real so I think that the harm to me would likely be lower than the harm to society if building websites were impossible.

The people who are sending the pictures are criminally liable, regardless where they got them.The fact that somebody built a website for it is irrelevant, the act of sending them unsolicited is the immoral act here. (And frankly it's probably gonna be laughed off or end up as spam unless somebody you associate with is an idiot.)

The goal is not to prevent someone from making their own model do what they want, but to prevent your model from doing what you don't want, like generate CSAM or non-consentual sexually explicit photos.

I don't understand how its controversial that someone/some-company might not want their products to be known for that.

It's a bit odd requirement, but.. OK. I mean who is the malicious actor here, the AI, the human user or the AI provider?

If the AI, then we shouldn't give it an agency (user should always vet the output). If it's the user, the AI is irrelevant to the question. And if it's AI provider, why would they train AI on such materials in the first place.

The whole enterprise of this kind of safety doesn't make much sense to me. If the AI is not able to follow so clear user instructions it's not ready for prime time and must be under human supervision at all times. (And subtle hallucinations on topic seem to be both more dangerous and more of a problem than blatant random production of explicit images, anyway.)

> It's a bit odd requirement

It is a realistic requirement for dealing with user generated content at scale, because this is a realistic requirement thing some people will do.

We’ve seen this fold out over the past month from Grok.

Reply on news.ycombinator.com