I don't understand how you can look at where AI is at right now and conclude that we don't need to worry about far fetched scenarios. Do you not think that someone a few years ago would have called current AI capabilities far fetched?
As for regulation, what would you even regulate? How would you regulate it? If you don’t have clear ideas for what you want, then wanting something currently unnecessary like regulation is only asking for trouble.
Look at microplastics. They’re in the entire food chain; their “far fetched scenario” is we accidentally sterilize every chordate on the planet, and we die because there’s no food.
Or global warming. The entire planet drowns, dehydrates, overheats or freezes.
Or COVID vaccines, this is more or less the same tautology as people that refused vaccines. They didn’t do enough to detect and prevent far fetched scenarios, people didn’t want to take the vaccine.
I’m not saying some regulation wouldn’t be good (though I suspect the aspects we want to regulate differ). I’m just saying it falls into a broad category of “stuff we should deal with, but aren’t” and it’s not even really at the top of the list. We have some things already in motion we need to deal with that overrule things that we aren’t even sure will ever happen. It’s a hypothetical existential risk in a world with extant risks we are ignoring.