I'm not arguing about the consequences or responsibility. I'm asking which humans would be capable of stopping the development of AI at this point in history.
It’s difficult to understand where this all is coming from because it’s so strangely disconnected from history. It’s not even consistent with current world events. The US is in a war with Iran over nuclear proliferation while also worrying about North Korea’s nuclear program. China and the US are in a trade war and Taiwan is under threat due to semiconductor capabilities (among other things, obviously). Open weight models allow anyone in any country to do AI work on their laptop. The top story on HN yesterday was a small team of academics making some cool discoveries on improving LLMs without frontier lab level compute.
How anyone can see all of this, in addition to decades of history of attempted nuclear diplomacy and wars, and posit that the whole problem is easily solved by simple agreements is confusing to me. The confidence with which these alternatives are being proposed and the unwillingness to acknowledge second order effects (like AI researchers moving to a different country and continuing to work) is baffling.
But yeah, there are a lot of comments from genie-rebottlers that are not very grounded in reality. They're functionally equivalent to saying "hey, everyone in the world, let's not actually use electricity after all" in 1905.
If we're talking game theory, a better model is the Stag Hunt [1]. At some point the equilibrium strategy is to cooperate.
Another difference is that real life is an iterated game, and the iterated prisoner's dilemma rewards cooperation.