Agree 100% here, with the assumption that the system is healthy to start with. And it's safe to assume it to be true for startups, which is the focus of PG's tweet. (If it's not, that start-up is already dead or close to be)
For startups, a competitive advantage is to do things that don't scale while you can. This includes "O(n²)" non-scalable communication, since n is still very small. And that kind of communication obviously works better in-person.
Late stage companies where async communication and slow-to-move things is the norm (due to the switch to a scalable communication method) actually benefit from remote work if the in-person office was kind of unhealthy for some people. Unhealthiness can be from the unavoidable slowness of communication/bureaucracy in a 50k+ employees company, or it can be from a rotten local culture in the office closest to you.
For this type of companies, remote options makes sense, the same way that expanding to other regions made sense. At this stage your communication is by nature scaled up, slower, and was already spanned across multiple buildings in original campus, so why not remote?