Ask HN: What are the biggest challenges/annoyances with working on remote teams?

Those of you who run or work with/as/ distributed/remote teams. What are the biggest challenges/annoyances you face in your day to day work?