Everyone who has made it up the ladder knows this and it's the single aspect that differentiates people who routinely get overlooked for promotions from the individuals who steadily rise through the ranks
A lot of politics gets much harder in remote work, and this is what these people are best at.
It's not productivity, it's politics that is much harder to do when remote working.
They also need to keep an eye on developers and need to feel thigs out, the relationship, their management status known and so forth.
That's very hard to do remotely and they feel their status diminished to that of the "resources" they manage.
That assumption has been thoroughly refuted by Google's reaction to James Damore's internal memo.