A more generous take is that managers used to derive a great deal of value & information from quick/frequent/informal check-ins with a lot of different people throughout the day, and that is much harder/impossible in a remote first work environment where all interactions must be explicitly planned/scoped.
In another context: it’s the difference between an educator walking around and monitoring their students progress all the time via conversation/q&a vs sitting behind their desk and only getting progress data via weekly assessments. The former is supplemental to the later and provides a great deal (perhaps most) of the educator’s understanding of their student’s progress. (The relationship between workers/managers and teachers/students is not analogous in many other ways, of course)