Hacker News Reader

Comment by femto | Hacker News Reader

Another bit of code in this space is libmv: https://github.com/libmv/libmv.

libmv's codebase seems to be forked, with an earlier version at goggle code: http://code.google.com/p/libmv/ which also contains an interesting summary of other libraries in the 3D reconstruction space. Blender also has its own fork, which it uses for matchmoving, which is the integration of animated objects into a real world scene.

In turn, libmv seems be be influenced by the work of Marc Pollefeys? The tutorial is a readable summary of how to go from a collection of 2D images to a 3D model.

http://www.cs.unc.edu/~marc/research.html

http://www.cs.unc.edu/~marc/tutorial/

http://www.cs.unc.edu/~marc/tutorial.pdf

Question: Can a knowledgeable person here suggest which codebase is the best to start experimenting with, to build an application that converts a 2D photo sequence into a dimensionally accurate 3D model?

View on HN

Sure! My lab focuses on SLAM and 3D reconstruction, especially for robotic applications. We've developed a BSD-licensed C++ (w/ MATLAB wrapper) library with specific applications towards 3D reconstruction problems such as SLAM and structure from motion. It's called GTSAM [1]

We actively maintain and release new features as they are published. While we don't provide a full out of the box pipeline (yet!), there are plenty of examples and documentation which walk you though the math, implementation, and other issues. If you want to read about the graphical models underlying GTSAM, see [2]

Utilizing OpenCV for feature detection and association is pretty much all you really need to add to a program in order to recreate Photosynth using gtsam. I'd also you recommend KAZE features from a former post-doc out of our lab, it's state of the art and recently added OpenCV wrappers[3]. However, it's also trivial to integrate other sensors such as IMUs, GPS, lasers, etc. for full navigation problems.

If you wish to know more about the actual subject, I definitely recommend Hartley and Zisserman's Multiview Geometery Book[4]

[1] https://borg.cc.gatech.edu/borg/

[2] http://www.cc.gatech.edu/~dellaert/pub/Dellaert06ijrr.pdf

[3] https://github.com/pablofdezalc/akaze

[4] http://www.robots.ox.ac.uk/~vgg/hzbook/index.html

do you think it would be possible and/or worth it to reconstruct a automotive road course? this is something i've always wanted to do. there's a ton of video online (though usually shot from inside the car.)

the one i spend the most time at is pretty flat, though. i suspect the green hills are hard to get a match on? here is an example: http://www.youtube.com/watch?v=EyZcERAlBeE

I often go on rides around mountainous areas on my motorbike with a video camera attached to my helmet (some videos here http://www.youtube.com/user/slashtomeu/videos ). Since I carry a GPS as well, I've always wondered if I could reconstruct a 3D model of the landscape I ride around in.

You need to come & talk with openstreetmap people more.

Although I don't have first hand experience with the code, a former labmate of mine (I'm at the Univ. of Washington, and he's now at Google) is one of the leading experts on this area of research, and his VisualSFM [1] tool is, I think, the best and easiest-to-use available online.

Briefly, there are three main steps required to go from images to a 3d viewer like PhotoSynth:

1. Figure out where each image was shot from (the "camera pose") and get a sparse set of 3d points from the scene. These two are estimated simultaneously using bundle adjustment [2].

2. Go from a sparse set of 3d points to a dense 3d model. This is done using a technique called Multiple View Stereo (MVS), of which the leading (open) implementations are PMVS/CMVS [3,4].

3. Build an image-based rendering system that intelligently blends between the 3d models and images to minimize artifacts.

The VisualSFM software will do steps 1 and 2. Step 3 is still quite a challenging problem, but depending on what you're doing, you could use standard 3d modeling environments to look at your data.

[1] http://ccwu.me/vsfm/

[2] http://en.wikipedia.org/wiki/Bundle_adjustment

[3] http://www.di.ens.fr/pmvs/

[4] http://www.di.ens.fr/cmvs/

A good place to start is the Photogammmetry forum, there you find info about photosync, bundler, PMS, etc:

http://pgrammetry.com

Reply on news.ycombinator.com