However I'll push back on this. If it's okay for the government to fund basic scientific research where the benefits are long-term and unpredictable, why not private individuals? Or are you saying research shouldn't be funded at all?
While I think it's important to fund such research, it's a good point that funding orgs should either be evidence-based or not. Funding some things based on evidence and other things based on hunches might be an unstable combination. Once hunches are allowed, they could easily drift into funding things where the evidence says it's ineffective, but it gets overruled.
Since AI can kill 7,000,000,000 people, You could probably toss a non-zero probability on it. Any math people know how to model this?
Even with a bunch of different variations for the probability, you could understand why AI is so dangerous. potentially.
There are probably lots of non zero probability events like getting hit by a rogue quasar. The difference is we cannot stop the quasar and that humans are required to discover AI.
I dont give money to other charities, I spend money on mine. We dont have any employees, just volunteers. I spent a total of 50$ on the LLC paperwork and 8$/mo for a website host. Everything else has went directly to the kids/families we are teaching.
For estimating the likelihood of AGI, you can start with estimates from people in the field. Some say 0%, others say 80% chance in 50 years. Any way averaging of these produces a risk high enough to justify serious funding into safety research.