Term limits would also remove anyone with any potential to collect enough power to change the system.

And there is nothing that prevents the replacements from abusing the power and control afforded to them in similar or worse ways. They'll be kicked out regardless of what they do, so why care about the long term?