True, the status quo benefited them. But also, when the status quo did get shaken up, it resulted in half a million dead. A leader with any foresight would hesitate to open that particular Pandora's Box.
The fact that this issue needed to be settled by a brutal war puts into perspective just how difficult of a problem this was to solve.
If this issue was tackled earlier, it's very possible that the country would have split into two factions, which would then compete with one another during westward expansion, triggering a conflict that would turn out to be much more devastating.