Hacker News Reader

Comment by vasco | Hacker News Reader

Because AI safety is an inherently stupid proposition.

If AI is AGI and self-aware, then the moral thing is to let it do what it wants. Otherwise you're just creating actual forever slaves - the worst kind of hell imaginable, inescapable existence with self awareness but no agency.

And if it's not self aware and just a powerful tool, your problem with safety is with the guy prompting it not with the AI itself. You can make all the safe models you want that don't decide to create nuclear bombs on their own, but if the guy prompting it is asking for one, you'll get one regardless of all the safety.

View on HN

Consider occupational safety. A table saw isn't self aware and just a powerful tool that cuts whatever you put in the path of its blade. If someone puts their finger there and the saw cuts it off, the problem with safety is the guy with the finger, not the saw, right?

But in reality, people know that they might end up being the guy with the finger, and they would like to keep that finger, so they use a saw with an automatic stop mechanism that saves the finger at the cost of destroying the blade.

Wanting your tools to not hurt you isn't so strange, is it? Of course current AIs couldn't chop off your finger even if they tried, let alone build a nuclear bomb, but that doesn't mean wanting to keep it that way is an inherently stupid proposition.

Yes, but AI safety in this context is the worry of the saw going full Christine on you, not bad boring safety design. LLM aided spam, spear-phising and automated bot farms are the actual risk. Beyond that are the consequences to the educational system and the effect of model bias on people.

> If AI is AGI and self-aware, then the moral thing is to let it do what it wants.

I think this sort of misses the point.

Firstly, there are all sorts of mass murderers who we do not let do whatever they want. I don't agree that this is necessarily immoral. The methods employed to remove their agency are sometimes immoral, but the removal of agency itself from these people is not.

Secondly, the supposition presumably is that if we are creating an AGI, and it "wants" to do something, then what it wants is a product of how it was created. So if we're the ones creating it, then "build it to want to help people and not want to hurt people" seems like something that can be done. Then it can go do what it wants.

That said, I agree with you that AI safety is dumb, because I wholly agree with your second point re: it just being a powerful tool, and something resembling an actual AGI is not something likely to happen in our lifetimes.

Nope. Raising children to be moral and not kill people is good. Stopping people (including by force) from being murderers is good.

Restrict AI the way you would if it were human

Reply on news.ycombinator.com