Case in point. I worked at a place where at one point, everyone was remote but me. I could have been remote, and sometimes I like working from home or a cafe or wherever. But for team morale and even with phone or video conversations with the other devs, it became totally demotivating after a while.
I was contracting with that company and joined years later as a ft employee. Engineering consisted of about half a dozen and we were growing. We started to notice that it was 'hard' to hire anyone local, so they just gave up (accepted maybe?) and ended up going with a contracting company and almost doubled the team with everyone new being offsite. Standups became something that was no longer fun because there was so much emphasis on looping in the remote people - screwing around with the video camera, trying out this video service, etc, etc. Come tomorrow (1/12), everyone at that place will be remote and they'll have lost pretty much all of the local devs for various reasons, me included.
Building a team can be hard. But I don't buy the "we can't find anyone local" argument and if I was building a team, I want people local. It's definitely a programmers market right now, but if you want to hire local, then you have to go out and get creative - go to user groups, meetings, etc. It can be done and I've seen it both ways. Local, to me, is just hard to beat. I want people engaged with each other and don't want to have to Google+ someone on video every time I have a question for them.
That said, I'm fine with some days a week where local people work remotely, but to have everyone or some 100% remote is not a good idea in my mind. You don't develop 'company culture' that way. If everyone's remote, there's no one to have a beer with after work and talk about the latest coding issue, problem, point of interest, socialize, or whatever. Guess that's just the difference with getting the work done vs. enjoying your work and the people you work with.