I'm with you there, it just happens not to be the same with everyone.
Some folks have very complicated home situations, living with family they don't get along with, living with roommates that are fun, loud, and work for competitors, living with boyfriend/girlfriend that they're having a falling out with, etc.
Some folks have demons of their own that they can't deal with without social pressure. People with alcohol abuse problems for example, since nobody can smell your breath on videoconference, might have trouble not day drinking when they're at home with their stash all day.
Finally and perhaps most importantly, it is a very different experience for kids coming out of college or very fresh in the industry not to have any folks they can easily turn to for advice they'd otherwise have gotten from work colleagues. Work advice, career advice, life in the city kind of advice, you name it - especially if you moved cities or countries for your career, the workplace is a really important factor in helping you get settled.
I'm not saying on balance these would tilt things in favor of working from home or the office, but I'd encourage you to ask around coworkers how they feel in order to get a sense of what other situations might be like.