You are not alone. We often under estimate the value of workplace bonding. Planning a dinner for the team when we have been through a tough development cycle for a couple months for instance is incredibly rewarding. It vastly improves the quality of collaboration and the way people engage with each other. Helping them communicate in an informal settings and find better ways to connect with each other even on an emotional level.
Remote work takes all of it away and adds this invisible burden to navigate through your teams thought process. That alone is a huge factor for me to make sure there is a balance between remote work and office visits.