The alignment problem is largely incoherent. Aligned with whom? Most problems will require unequal and arguably unfair sacrifices.

Further, people even obvious and largely inconsequential solutions are controversial. If an AGI suggested wearing a mask during a pandemic that spreads via airborne particles, a large percent of the population would say the AGI is “unaligned” and we’re living through skynet.