You're clearly more towards the extrovert part of the spectrum. Please try to have empathy for people on the introverted part of the spectrum who cannot fathom being forced into an office with tons of people distracting them all day every day. Heretofore they (save for a lucky few like me) have been forced to not just fathom it, but live it.
The "half" of Americans mentioned in the article probably has heavy overlap with those introverts.
Honestly, your post reads pretty heavily to me of "status quo privilege bias". You fit the current status quo very well, and are expressing worry that a change to that status quo would hurt you, with out extending empathy to those the current status quo hurts badly.
The world is not going to flip to fully remote work. It's likely going to inch towards remote work being more prevalent or available. What you are expressing is a case of "When you're accustomed to privilege (your way of working being privileged in this case), then equality (remote work being equally available) feels like oppression."
Not saying that makes you evil or any thing, just pointing out that this is a pretty standard human bias that gets expressed and winds up working to defend harmful status quos all the time. Worth being self aware of.