It's a nice thought, for sure. I think people would generally use it only to hang out and relax with their friends though. People, for the most part, don't want to sit down and get into an intense discussion/debate over controversial topics with someone from "the other side". They'd much rather be around people with whom they agree.
I could see it being useful as something of an intermediate between traditional online video games and offline gatherings involving food & boardgames. I don't see it being used as a primary mode of communication though, for the simple reason that it demands too much time and attention. Text messages and email occupy that space because it's so convenient to pick up and drop them at any time, even with a half-read message or a half-written reply. With VR that means putting on your gear every time, and resuming the "VR video" or whatever.
Heck, tons of people complain when video talks are posted without slides/transcripts. They want the information. They don't care about any other aspects of the presentation. VR in that case just seems to be more distracting stuff that gets in the way of the information.