Have you used a vr headset? It's great for gaming if you want to experience immersion, but for everyone else it's like being blind folded (not ideal in a public or office environment).
No matter how good the hardware gets, vr will not reach mainstream audiences for a similar reason 3d glasses didn't. No one likes putting technology on their heads or faces. Spectacles are an exception.