http://13thlab.com who got acquired by Oculus made something very very similar:
https://www.youtube.com/watch?v=rA2NI4NgsV0&t=19
so the technology for doing this is feasible and real.
Their previous kickstarter: https://www.kickstarter.com/projects/1203948132/rescape-fps-...