Hacker News Reader

Comment by crystal_revenge | Hacker News Reader

> that's governments job

Why do people take these AI "safety" research projects at face value? The real reason you need AI that is "safe" and "governable" is so that when you start having it promote advertisers content or support the current administration, you don't have to worry about it going "off the rails" and promoting a competing product to criticizing the administration.

I'm sure plenty of researchers in this space also believe they are working for the good of humanity, but I suspect the real am is much more practical and perfectly aligned with the business interests of all the companies sponsoring this type of work.

View on HN

We currently can't do "AI safety" even in bleeding edge alignment research so investing into some startups in that area is just burning money. Current LLMs/ViTs have non-zero probability of producing something unsafe and it's their inherent trait.

> Current LLMs/ViTs have non-zero probability of producing something unsafe

Show me on the LLM where the "unsafe" result is that doesn't already exist inside its training data set? Go ahead I will wait.

Just so I understand your comment, are you trying to say that the money on safety should go first into research? And not into a later startup, which is just putting lipstick on the pig? I agree to this understanding, no doubt about it.

Basically yes. First figure out how to do reliable models that can be well-aligned with safety expectations, then invest into a startup that brings that tech to the industry.

Uh, I mean, I think we've seen first hand that generally one aspect of safety we want is to not produce sexualised images of children.

Is it even an attainable goal? It seems an NN with less than say 4 billion parameters will be able to do that. The cost of training will likely go down with more models being available. Unless we lock down the computing for majority of people, I don't see how we can prevent someone creating a CSAM model in their garage.

I don't want to see CSAM created, but the totalitarian control required is too much for my taste (and frankly it's preferable for that person to use NN than to go out and hurt actual children).

Not to mention even locked down technology is often being abused by the privileged.

People can draw naked children with a stick and some dirt. I’m not sure that preventing the creation of fictional csam is the best use of our resources if we want to protect minors from abuse.

The best use? Probably not. But if I built a website that let people generate extremely convincing unlimited photos of you wearing an SS uniform and forcing your dog to smoke meth and sent them to everyone you’ve ever met, this might seem like a less worthy hill to die on. Or is that just a sticks and dirt thing too?

Oddly specific... Did that really actually happen to you?

Everybody I care about would know that those pictures are not real so I think that the harm to me would likely be lower than the harm to society if building websites were impossible.

The people who are sending the pictures are criminally liable, regardless where they got them.The fact that somebody built a website for it is irrelevant, the act of sending them unsolicited is the immoral act here. (And frankly it's probably gonna be laughed off or end up as spam unless somebody you associate with is an idiot.)

The goal is not to prevent someone from making their own model do what they want, but to prevent your model from doing what you don't want, like generate CSAM or non-consentual sexually explicit photos.

I don't understand how its controversial that someone/some-company might not want their products to be known for that.

It's a bit odd requirement, but.. OK. I mean who is the malicious actor here, the AI, the human user or the AI provider?

If the AI, then we shouldn't give it an agency (user should always vet the output). If it's the user, the AI is irrelevant to the question. And if it's AI provider, why would they train AI on such materials in the first place.

The whole enterprise of this kind of safety doesn't make much sense to me. If the AI is not able to follow so clear user instructions it's not ready for prime time and must be under human supervision at all times. (And subtle hallucinations on topic seem to be both more dangerous and more of a problem than blatant random production of explicit images, anyway.)

> It's a bit odd requirement

It is a realistic requirement for dealing with user generated content at scale, because this is a realistic requirement thing some people will do.

We’ve seen this fold out over the past month from Grok.

> promote advertisers content ... promoting a competing product

Where is the value of AI when the responses are compromised like this? I could say the same thing about Google Search, which is one of the reasons I stopped using it.

Are we betting on the masses not caring that they're being lied to for profit?

Reply on news.ycombinator.com