Agreed. There's something fundamentally different about contacting someone via a video conference versus the organic nature of a quick chat about something.
This puts remote workers behind quite a ways. My old team was partially located in NYC and partially located in the Bay Area. There was a major difference in inter and intra location interactions and it affected the team in some negative ways.