Operating systems are less regulated than milk, which is why they blue screen without consequences. Let's not entirely lose the plot here.
Think of it as a guard rail that actually works.
Imagine the fortune food supply companies will make! “Our lettuce just might poison your child!”
Truly a golden age is near, and you, perceptive geniuses that you all are, will be the only ones not fooled, not for a second, because it all makes sense, always and forever, as long as you squint hard enough to close your eyes.
I've not seen anything so far that's remarkably more dangerous than a laundry list of commonly purchased items. Most of what I hear is either some sort of far fetched "the teenagers are going to be bio-engineering designer strains of polio", or some vague fears that the AI will become sentient and kill us all that vaguely reminds me of when the Manhattan Project thought they might cause earth's entire atmosphere to enter runaway fission.
I'm also distrustful because virtually all the experts stand to benefit from convincing us unregulated AI is dangerous. Namely, it's a compelling reason to ban Chinese and open weight models that are eating into their margins. It doesn't mean they're wrong, but it does mean that they'll need to substantiate their claims. I'm not trusting them at face value.
What does Geoff Hinton have to gain?
This is a conspiracy theory FYI.
[citation needed]
That would be bioweapons. Nobody's worried about them, regulations and treaties are conveniently and totally dysfunctional. Welcome to the real world.
Nukes could kill us all, unhinged people have access to them. Global warming appears inevitable, and we’re trending towards “world ending”. I don’t recall how many dinosaur killer asteroids have a path whose margin of error includes hitting Earth. We’re about due for a “fry all the electronics on earth” solar flare by probability. Microplastics seem okay-ish so far, but they’re in the whole food chain. If they somehow end up causing sterility at certain concentrations, we might kill our whole food chain.
No, it’s not that notable that some technology could theoretically, one day, in ways we don’t currently understand, kill us all. That’s been basically a constant since rhetoric Industrial Revolution and our ability to blanket the earth in a particular thing.
I could go buy a car in cash today with 0 paperwork. It only has to be titled and registered if I drive it on public roads. Even that’s like 90% a technicality. I drove a couple thousand miles on tags that were 6 years out of date (didn’t drive it but moved and had to move it).
Most states also let you drive on dealer plates for 30 days or so. Steve Jobs notoriously got a new car every month so he never had to have real plates.
If the presumed goal is “acquire a car and do bad things with it immediately”, then no, there’s no regulation preventing that. You can buy a car off Craigslist and play bumper cars on the highway 5 minutes later without ever talking to the government or proving anything.
Basically the entirety of car regulation is to protect normal folk from normal problems, not preventing the insane from doing insane things.
As for regulation, what would you even regulate? How would you regulate it? If you don’t have clear ideas for what you want, then wanting something currently unnecessary like regulation is only asking for trouble.
Look at microplastics. They’re in the entire food chain; their “far fetched scenario” is we accidentally sterilize every chordate on the planet, and we die because there’s no food.
Or global warming. The entire planet drowns, dehydrates, overheats or freezes.
Or COVID vaccines, this is more or less the same tautology as people that refused vaccines. They didn’t do enough to detect and prevent far fetched scenarios, people didn’t want to take the vaccine.
I’m not saying some regulation wouldn’t be good (though I suspect the aspects we want to regulate differ). I’m just saying it falls into a broad category of “stuff we should deal with, but aren’t” and it’s not even really at the top of the list. We have some things already in motion we need to deal with that overrule things that we aren’t even sure will ever happen. It’s a hypothetical existential risk in a world with extant risks we are ignoring.