A 90% Chance of AI Saving Humanity
joshbocanegra.medium.com
joshbocanegra.medium.com
Both figures are totally fictitious and have no bearing on reality whatsoever. They tell us about the hopes and fears of those people citing them, but aren't otherwise informative or useful.
That's essentially what "effective altruism" does to these people.
If you haven't seen the movie, sorry, it's a giant omnipotent artifact that causes all kinds of horrors by just manifesting what nearby humans think strongly about.
AI seems to be dangerous precisely because we tend to think, write, and test it as dangerous. What if there's a lot more "expectation fulfillment" going on with these models than people might appreciate, by virtue of their fundamental architecture perhaps.
Don't get me started on people "asking AI what it wants" and getting obvious scifi tropes out - the question answers itself when you frame it that way.
Think of it more as it is reasonable to say there is NOT a 100% chance that AI will save humanity, i.e. don't count on it. In the same way as it would be unwise to count on there being a 0% chance of AI killing us.
Whether or not the 90% (optimistic focus) or the 10% (pessimistic focus) numbers are accurate is less the point. But the shape of the distribution and how to navigate that is the bigger thing to think about.
An important thing to keep in mind is that 10% in which AI destroys humanity is a scenario in which the AI goes rogue and decides to destroy humanity of its own volition.
I think if we're being more realistic, there's a much larger chance that AI destroys humanity at the instruction of its masters. Once AI can automate enough industry, silently distribute IFFs among the Epstein class, then send out drones to kill any humans without.
We can't always pin the risks on the Chinese, so imagine a scenario where a disturbed insider at Anthropic smuggles out the weights for Claude Aprocrypha and runs them through abliteration. This is not super expensive to do on rented hardware. Then the disturbed human tells the refusal-free model to do anything necessary to launch nuclear weapons.
It'd probably only kill a few billion people though, so not really an "existential threat" in literal terms. :-)
If a model on rented hardware could manage to launch some country's nuclear weapons, the same could be done by model-armed or model-free humans, spies, etc from other countries - using their specialized tooling they could always do better than any model.
In short, AI doesn't add any new threats to the mix, the defenses against AI are the same as against adversarial humans, and the whole brouhaha is nonsense. Sure, make your defenses impenetrable, use AI for that and it'll stop any I, not only AI.
I agree the solution is to fix the vulnerabilities. I think "impenetrable" requires a radical shift in popular mindset about operating systems, applications, and devices connected to the network.
But that's wrong. If the doom scenario is 10% likely, that in no way implies that the utopian scenario would be 90% likely. They're both at the extreme edges of the bell curve, so should at least have the same made-up odds as each other.
The reason I pointed out the over-the-top BS nature of the 90% figure is because it indicates that the essay is a propaganda piece and should be treated as such, rather than any kind of serious think piece.