There are Intel Realsense cameras, available off-the-shelf today, which provide stereo vision complete with on-device processing, IR texture projector, and a built-in accelerometer.

If you want to play around with stereo vision and figure out the feasibility of your idea, it could be a good place to start.