It's easy to think that AI is mostly a harmless concept, after all, all it does is give you text and image responses, how can that cause any serious harm? But that betrays a very serious blindness to how quickly these things can change, it's like thinking the invention of electricity cannot possibly have any huge implications for society because all it does is light some lightbulbs, and we already have oil lamps that does roughly the same. The future is not simply what you have right now scaled larger.

A lot of money, time, and effort is being poured into AI, and putting anything but a trivial chunk of those resources into AI safety, or limiting further development of AI until you know it's safe, means that you'll lag behind your competition who aren't doing such pit stops.

If OpenAI, Google, or Meta decide that they won't release AI models or services into society before they know it won't do serious harm in some unexpected way, then the other two will speed ahead and capture the momentum and market share.

As usual, capitalism pushes us ever closer to the gaping maw by it's very design. If you aren't willing to entertain the destruction of society in your pursuit of new innovation and technology, you'll be out-competed by those who do.