Automatic camera calibration refinement/healing. SfM is a well developed area, and there are well understood methods for gathering correspondences like SIFT in order to calibrate cameras. However, it usually requires a separate calibration phase prior to doing whatever you want to do with that calibration. In my case, we’re using it to triangulate 3D objects from 2D object detections done in each of our calibrated camera.

Over a long time period, some perturbance befalls those cameras: gravity pulling them down, things knocking them, etc. That causes our accuracy to degrade over time, so we started looking at ways to continually heal that calibration without needing to do another stop-the-world calibration phase.

Turns out you can use the 3D objects themselves (actually the 2D constituent detections that you use to compute a 3D point) as correspondences instead of doing SIFT matching. The hard part is that you have to preserve the original coordinate frame after doing a bundle adjustment.

Computer vision is a ton of fun, and I highly recommend you give it a try if you’ve enjoyed data science stuff, but the incomprehensible “try it and see” engineering of working on deep neural nets turns you off.