The sweet spot in my experience has been for WFH orgs to get together periodically for extended offsites. A one week gathering twice a year has forged some incredible bonds on my distributed team.
of course I got hired in that place in 1/2020, so we know how that turned out.
I'm the only person "remote" (work from another office/country) in my team and I am the one that always has to travel to meet the rest of the team.
It's nice meeting people and all, but I generally find these offsites pointless and not work the hassle of traveling for it.
All of your complaints are (usually) easily solved by experienced travelers. It's okay to not like travel, but it doesn't condemn the concept of a weeklong company offsite a couple times a year.