Would there also not be some reputational damages that could be compensated for as well? These are the types of cases usually ending in a settlement for non-disclosed amounts with NDAs attached.

I feel like it's the government that has to sue OpenAI for reckless endangerment of human life, or something similar. I don't believe the current models could "take over the world" even if humans tried to help them, but both OpenAI and Anthropic themselves talk publicly about how dangerous they believe these models to be. Then if that's so, they should be held responsible as if the models were that dangerous, but who's supposed to put that in motion? The only entity I can think of is the government.