I'm fine with people coming in maybe 3 instead of 5 days a week. I'm not fine working together with people who are not here at all. It's wildly inefficient and communication via video conferencing is horrible.
Nobody cares about individual preferences as in the end we vote with our feet. So you not wanting to work by yourself should have no bearing on what others in the same group do. Once you start mandating what others should do based on your needs, people will either not listen, leave or you will leave.
True, I do vote with my feet. Which is why I won't apply for jobs with remote teams, and which is why I'm sticking with my current job.