> Someone who hasn't been reading the object-level arguments for 10+ years might reasonably wonder if these are good heuristics.
I think you might have meant hermeneutics, in the sociological sense of 'subjective interpretation'?
I think you might have meant hermeneutics, in the sociological sense of 'subjective interpretation'?
Yudkowsky and friends believe many things, and reasonable people can disagree about many of them, but "at least 10% risk of extinction if we don't slow down" is a relatively straightforward conclusion based on the offensive capabilities of unreleased models and the terrible state of biosecurity and cybersecurity.
Even without AI, the risk of extinction might be above 10% if people with the relevant skills were interested in using them that way. AI adds an alien willingness to achieve poorly specified goals at any cost.