I don't know you or your coworkers, but dang, trash talking them on HN? Sounds like a great place to work </sarcasm>. Definitely sounds like some communication issues going on.

I've worked for the past 8 years on remote teams, and from my experience, having a shared mind is very important to a highly productive team.

My suggestion to you would be to communicate with your coworkers that you've already talked about your weekend with so-and-so, building resentment with your coworkers is never good. Maybe even create a channel specifically to talk about your weekends! Who knows, you may like it. :)