> "shut it all down" being the only course of action
And how would we “shut it all down” in other countries? War? Economic sanctions? Authoritarian policing of foreign states? Enforce worldwide limits on the power of GPUs and computers?
And how would we “shut it all down” in other countries? War? Economic sanctions? Authoritarian policing of foreign states? Enforce worldwide limits on the power of GPUs and computers?
https://www.lesswrong.com/posts/oM9pEezyCb4dCsuKq/pausing-ai...
Basically, the idea is that countries sign the agreement to stop the large training runs, and, if necessary, be willing to use conventional strikes on AI-training datacenters in the countries that refuse. Hopefully it doesn't come to that, hopefully it just becomes the fact of international politics that you can't build large AI-training datacenters anymore. If some country decides to start a war over this - the argument is that wars at least have some survivors, and an unaligned AI won't have any.
As a "highly intelligent system", we have a long history of extincting animal species, and we're well on track to eventually extinct most of them, despite it's highly likely this will make Earth uninhabitable for us, ending with humanity dying off too.
Why do you think AI can't casually extinct us, because it doesn't need us (or thinks it doesn't), and we're just in the way of whatever it is that it wants to do?
Well, we care for various reasons, major one being our own survival and comfort. Given what that means in practice, we'd be better off dead than having an AI care about us like we care about animals and plants.
All things the USA has already done.
And, the "AI-might-end-up-killing-everyone" community doesn't seem to be able to see this through other people's eyes in order to make an argument for this without belittling the other perspective.
If other people change their minds, it probably won't be through persuasion but from catastrophe.
Why settle for the maybe-catastrophe of AGI when we can have the definitely-catastrophe of world war?