We live in a society experiencing a crisis of loneliness. Stripping away yet another built-in opportunity for in-person socialization and community building is going to make that worse, not better.
I get that all it takes is to find a meet up of like minded individuals that meet up regularly, but sometimes even that is hard to find.
You left out the military, but those are merely the few institutional walls that society has most oppressively forced upon us. There are countless other sports teams, game clubs, hobby communities, activist groups, charities/service organizations, hacker/maker spaces, martial arts gyms, etc. etc which we are more free to choose from, and are often more likely to enjoy. With the internet, such groups have never been easier to find.
Choosing is only an issue because we are so unused to having such agency over our own lives.
Coworkers offer a great pool of like minded individuals that can relate to people. You have to interact with them, and relationships are inevitably built. This is fantastic!
On a personal note: the office is a social environment distinct from my personal life. I enjoy both: hanging out with coworkers after work talking about a hairy problem, working together with coworkers to fix on call issues, and then hanging out with personal friends on weekends.
To be clear: I have mixed feelings about remote work. I can see the benefits it offers but I love to be around human coworkers. I’m not sure how invested I would feel.