Is that because many organizations tie together the concepts of higher pay/benefits and moving up in the managerial hierarchy?
Considering there is evidence that the Peter principle holds to a certain extent, that has always seemed odd to me. A person being best at one job is not a strong indicator that they'd be best at another. Personally, I have no desire to be promoted into roles that don't suit me.
And even if not - the folks higher in the chain also get to decide what gets to be worked on.
Making management the only option for having a "career" is the issue.
Another option to avoid the downsides of the Peter Principle is to test drive people in their new roles. Basically you slowly add responsibilities that look more like the post-promotion job, if they do well they get the promotion and they've already proven to be able to do these things. If at some point you notice it isn't working, well then you don't promote them, or promote them to a different position.
https://en.wikipedia.org/wiki/Skunk_Works
> The designation "skunk works" or "skunkworks" is widely used in business, engineering, and technical fields to describe a group within an organization given a high degree of autonomy and unhampered by bureaucracy, with the task of working on advanced or secret projects.
One of the key findings of the Skunk Works project is that you unlock potential, a company needs to be willing to pay individual contributors more than their managers.