Have you ever used a video conference or collaborative online whiteboard? If the actual scribbling is important, have you heard of these things called drawing tablets?
Its still, at least to me, markedly less productive, than an in-person meeting.
Like I said, conferences, summits, etc. These are done for a reason. Not to mention that there's some research I've seen that suggests that places like Bell were so successful because they cultivated happenstance interactions between smart people that don't happen in distributed environments.
My current belief is that a 95% remote team works, you do need in-person opportunities, the more the better, but at least once in a project can be enough.