I get this is Hacker News so we have to hate AI safety, but you should read more on this topic rather than blithely casting the x-risk viewpoint as fringe. The last hard data we have on the question is a late-2024 AI Impacts survey [1] of 1,580 authors at top AI conferences, who were offered $50 for answering a survey on the topic of the future of AI progress. The median respondent assigned a 5–10% chance (!) to "extremely bad (e.g. human extinction)" outcomes.
> On the factual end of things, AI safety experts have been raising the alarms for over a decade and so far essentially none of their predictions have come true.
Devastating. When I wrote my AI alignment PhD thesis [2] in 2022, and predicted a 70% chance existential risk by 2072 (which I later came down to about 1/3 on), you're right that that prediction hasn't come true yet.
While I agree many specific worries didn't materialize (like AI being like a "literal genie" it's hard to specify goals to), many key worries have (Hugging Face rogue AI swarm scans as cribbed from rationalist AI scenario writing).
[1] https://aiimpacts.org/wp-content/uploads/2026/09/ESPAI2024.p...
[2] https://arxiv.org/abs/2206.11831, top of page 4