Hacker News Reader

Comment by andsoitis | Hacker News Reader

I don't think you're meant to fixate on the number as being accurate or precise. It simply keys off of the "10% alternative", which is also made up.

Think of it more as it is reasonable to say there is NOT a 100% chance that AI will save humanity, i.e. don't count on it. In the same way as it would be unwise to count on there being a 0% chance of AI killing us.

Whether or not the 90% (optimistic focus) or the 10% (pessimistic focus) numbers are accurate is less the point. But the shape of the distribution and how to navigate that is the bigger thing to think about.

View on HN

> It simply keys off of the "10% alternative", which is also made up.

An important thing to keep in mind is that 10% in which AI destroys humanity is a scenario in which the AI goes rogue and decides to destroy humanity of its own volition.

I think if we're being more realistic, there's a much larger chance that AI destroys humanity at the instruction of its masters. Once AI can automate enough industry, silently distribute IFFs among the Epstein class, then send out drones to kill any humans without.

Lol, I suspect it will be humans instructing the AI that cause most problems too.

We can't always pin the risks on the Chinese, so imagine a scenario where a disturbed insider at Anthropic smuggles out the weights for Claude Aprocrypha and runs them through abliteration. This is not super expensive to do on rented hardware. Then the disturbed human tells the refusal-free model to do anything necessary to launch nuclear weapons.

It'd probably only kill a few billion people though, so not really an "existential threat" in literal terms. :-)

> Then the disturbed human tells the refusal-free model to do anything necessary to launch nuclear weapons.

If a model on rented hardware could manage to launch some country's nuclear weapons, the same could be done by model-armed or model-free humans, spies, etc from other countries - using their specialized tooling they could always do better than any model.

In short, AI doesn't add any new threats to the mix, the defenses against AI are the same as against adversarial humans, and the whole brouhaha is nonsense. Sure, make your defenses impenetrable, use AI for that and it'll stop any I, not only AI.

Reply on news.ycombinator.com