The assessment that it is more difficult to "have the managerial experiences and interpersonal experiences that you need to have to take your career forward in a work-remotely environment" is fair. Because that's office politics.
Also, it's fair that it makes the company culture fragmented and nearly impossible to move together as a team for folks without that experience. Silos are real.
But, in my experience, the main reason is that senior leaders cannot let go of the 'it's work time you should be working' mindset, and understand that some people work best in the morning, some at night, and some in small bursts with breaks between.
I have learned that a VAST majority of companies simply suck at setting goals and measuring employee productivity.