This just ignores the very real coordination problem. The signatories do not represent the entirety of AI development, nor do they want to unilaterally forgo business opportunities that the next man will exploit. Government is the proper place to coordinate these efforts, and so that is where they appeal.
It smells like bullshit being espoused to push an agenda, but I can't tell what the agenda is. My guess: play up the huge unrealistic risks in order to distract from the more realistic ones.
Two things about this tech that I'm personally not worried at all about: "evil agi" and "only the elites will control this".
There are so many things wrong with this line of thinking. First, it mischaracterizes the issues. Few people believe AGI guarantees the extinction of humanity. The issue is that there is a significant potential for extinction and thus we need a coordinated effort to either manage this risk or prevent its creation. It does little to stop the coming calamity to single-handedly abstain from continuing to build. Coordination is a must. Besides, most people will think they stand a greater chance of building it safely than the next guy. The coordination is required to keep other people from being irresponsible. Human hubris knows no limits.
The other mistake is misjudging nerd psychology. You can believe there's a high chance of what you're working on being dangerous and still be unable to stop working on it. As Oppenheimer put it, "when you see something that is technically sweet, you go ahead and do it". It is a grave error to discount the motivation of trying to solve a really sweet technical problem.
Ultimately these kinds of claims are self-serving, they provide rational cover to justify your predetermined beliefs that those calling for regulation are trying stifle competition. Folks don't want to be left out of the fun. The justification is in service to the motivation.
“Artificial intelligence could lead to extinction, experts warn” - BBC front page news headline earlier today in reaction to this story.
This is the problem. Reading that BBC article will make your average joe petrified. “Extinction” is a much catchier headline than the slow creep of automation replacing/changing jobs. The latter is literally already happening around us right now and it serves some of those signatories if those issues aren’t regulated against [1]. I’m not saying forget about extinction threat. Clearly that’s an important risk to manage, but let’s not ignore these near term, huge disruptions because policy makers are busy reacting to distracting headlines.
Edit: add ref; [1] https://www.reuters.com/technology/openai-may-leave-eu-if-re...
Misdirection. We see a generic call for regulation, or unrealistic call for a global pause with no answers to how it would be coördinated or enforced. When actual regulations are put forward, they're rejected without a counter-offered solution [1].
[1] https://www.reuters.com/technology/openai-may-leave-eu-if-re...
There is a difference between trying to shape regulation to your advantage and engaging in bad faith. There is no indication there is real regulation that addresses the problem these folks are brining up that they would support.
Many other people talking about x-risk signed the letter calling for a pause on training new big models, which seems pretty concrete to me. Does that make sense to you? If not, do you have other regulation you'd prefer?
It's totally unrealistic. How are you going to coördinate a global pause on AI research?
Nobody has even bothered proposing a framework because of how ludicrous it is for this to be what Beijing, Washington, Brussels and New Delhi focus on.
But it's not clear to me what you think would be a good idea, which makes it difficult to see what you think would be a serious proposal rather than misdirection. It appears to me that you are taking it as evidence of bad faith if someone proposes regulation either stronger (pause) or weaker (OpenAI amendments to proposed EU rules) than what you prefer - but you also haven't said what you do prefer. What am I missing?
Personally, while I don't think the pause is realistic, I don't think advocates necessarily have any obligation to propose realistic ideas when they're not in a position to actually write any laws and are only able to potentially shift the window of what politicians and media consider acceptable.
Registration. Not even licensing. If you're training a model, you file a form describing who you are, who you work for, who you're working with and what you're doing with which data for what purpose. This lets professionals, policy makers and the public get a sense for the landscape so we can make rules with some sense of reality.
Sure. Then the EU proposed something like it and Altman threw a hissy fit. That's where my accusations of bad faith come from.
Bad faith is unnecessary for bad outcomes. The perverse (from our PoV) incentives are sufficient.
OpenAI's incentives is to shut down all the rinky-dink models every hacker and their dog are building on cheap hardware. They spent billions, and are having the market taken from them.
Nobody has proposed any specific and realistic regulations.
The code for AGI will not be some monolith of software architecture. The code will likely be simple. This means that someone in their basement could build it. The steps to get there are challenging, though. A single person could have developed the transformer architecture. A single person could have used 4-bit quantization and developed an AI that is just as good as ChatGPT and have it run on their local machine.
The difficulties are figuring out the best 'needle in the haystack' to solve the problem. This requires research, and this process happens much faster if you have more people working on it. For years, many people did not put any energy into AI systems because the hardware was not here yet. The hardware is here now. The cat is out of the bag.