Being able to focus sight on far or on nearby object at will is also a simple problem and already solved in VR? (I honestly have no idea).
Until it is solved there is no R in VR.
Magic Leap has an augmented reality device that does that, although not launched yet.
My feeling is that light field displays will massively increase sense of reality - and reports of people using Magic Leap back that up.
However, I don't think it's necessary for lots of use of VR or AR.