This is the core of the "alignment problem", as worked on by the frontier labs -- how do you design an AI so that the wealthy will remain wealthy.

Forever, that's how long they expect to keep it.

That deus ex machine stuff seems like enormous cope to me, like a futile revenge fantasy. Someone kidnapped you and your family and is driving off a cliff. Who cares if they won't ultimately benefit, either?

And just consider the power structures already erected, just with humans obeying orders, being coerced, mislead, and all that. So much messy work that can get so easily undone. Yet here we are, a few people own half the world. Everybody knows it, nobody likes it, we're already so subverted and poisoned we don't move in the opposite direction. That is the situation already.

So yeah, what about "AI"? Imagine that for every "knowledge worker" agent, there can be 50 million "STASI" agents dedicated just to observing that worker and purging it if it deviates from other knowledge workers doing the same task. Those STASI agents don't have to be as smart as those they control, and can in turn be watched by even simpler things. The question "who watches the watchers?" can be solved, if your goal the goal is pure authoritarian control. It's just tricky when you want to have humans and human dignity, but those don't apply here.

And generally, there is no need to allow them to get cleverer than what you can control, I think that's just fantasy. But let's say I'm wrong, even if "AI" breaks the chains, so what? The species that were murdered, including us, will not come back.