For me it's mostly about ease of collaboration. I love hashing out math/design problems on a whiteboard. Even with good tools there's nothing that allows the same sort of natural, dynamic conversation. Meetings turn into a series of loosely connected speeches, especially when there's large numbers.

I also find that simple conversations take way longer, especially when there's disagreement.

Thirdly video calls are super draining.

It's also really difficult to get separation between work and life. I find myself doing home chores doing the day intertwined with work.

Unlike some commenter, the social aspect isn't that important to me, but there's definitely some camaraderie missing. In the office, I feel like part of a team that's working together to achieve something. At home I feel like one of a swarm of drones that occasionally communicate for information exchange. It's not very fulfilling.