I don't know why you'd want to do so for most TV. I like to idly code at 1/4 speed during average shows. Like if the cousins want to watch Moana for the 34th time, I'm not gonna dedicate my full attention the whole time, but I do want to hang out with everyone else in the family.
On the other hand, I've never wanted to pick up a phone while my VR headset is on my head. It's engrossing! But the downside of VR is that it's so isolating. There's no way for me to be half in a head-mounted display. No one can casually look over my shoulder, nor can I causally peek up from my screen when someone says something interesting.
I guess it's a little bit of a different story if it's just the centerpiece of family time, though..
Everyone I know has at least one TV. You can buy a 58" TV at Walmart for $298. That's less expensive than an Xbox or PS console.