I actually like to go into the office from time to time, but there isn't any point when no one is there. I spend a lot of time at various clients' offices, and I notice a correlation: the more opposed to remote work a company is, the more meeting-driven it is. It is as if management feels like it needs to do something with all these bodies taking up space in the office. I see small to mid-size companies sucking up productivity with endless meetings.
That said, I think my company has lost a little bit of interactivity with the remote work force. I'm putting together a Google+ Hangout I call "Engineers' Office Hours". The idea is that if you aren't doing something billable, you hop in and offer help to someone stuck on a problem.