our entire team is remote. it's really obvious when someone isn't at their computer. it's basically the only hard requirement of the job beyond the work product - be at your computer when you say you are going to be at your computer.
It's the same reason why almost every company programs in blub vs. lisp, haskell, erlang, etc. The skill level of employee can be hard to find who can do those things. Companies already feel supply constrained with engineers, now imagine you add the haskell requirement and you need to hire 1000 of them.
Startups can leverage this disadvantage sometimes by not having a statistical management requirement and start with a foundational advantage. There is also the case of using haskell killing your startup although because of tool problems, and big companies being able to make small special R&D type teams that can use Haskell, so it's definitely a balance.
I remember doing a quick fix for one of the attractions at the melenium dome - the core part of the work was working out on paper all the permutations and what should happen the coding at a terminal was the trivial part.
BTW the program correctly run first time and was delivered in less than a day.
For example, "water cooler" conversations. In a physical office these happen because people are conditioned to make small talk. That is not the case virtually, as it's easy to spend an entire day in a chat room with a co-worker and never interact with them.