To make a blanket statement like this shows a lack of judgement. He further states that people who are particularly exceptional should be allowed to work remotely.
This means that a contributor who is substantially more productive at home than in office should be forced to sacrifice their gain in productivity if they don't meet the bar for "exceptional".
Furthermore, while the total number of "hours worked" might be more at the office, everyone knows that the number of fake hours worked is also higher.
Finally, the nail in the coffin to his argument is the following: If someone can successfully pretend to work from home without any performance repercussions, there is clearly a lack of proper management in the company since performance is not measured by any proper metric of success but rather by their physical presence or hours worked. If an individual's productivity is found to be lacking due to their preference to work remotely, this should be communicated by the management chain and appropriate incentives/disincentives have to be put in place to guide the individual to perform at the level desired.
For roles that require you to be present at the location e.g. human resources or hardware roles, you can make those positions non-remote. Certain kinds of work like software development or design can absolutely be done remotely if the management is competent.
Lastly you can make an argument for innovation arising from the propagation of ideas and that this is more likely to happen if people are colocated. I think this is probably true but I also suspect that you can design a remote environment that allows this to happen even more effectively than in person. One idea that I think can work is to hold a brainstorming session with round-robin 1:1 discussions. You can form a fully connected graph of everyone's pairwise experience bouncing off each other which is more effective than the small k to many interaction that happens in meetings where an overconfident vocal minority comes up with most of the ideas.