Technically it is fairly easy, the bigger problem is accepting you cannot apply acceleration to the camera, and this messes up all sorts of assumptions.

It is certainly nothing like as hard as game development, but for people coming from TV/movie land the whole software and interaction aspect (even just moving the head) confuses them.

If anything resolving 3D audio was the hard part, by which I mean most subject to being misunderstood. (Try explaining to a media creative type that crossfading binaural mixes of two orientations does not create a valid intermediate new binaural mix).

Source: I worked on several notable stereoscopic VR productions.