Hacker News Reader

Comment by chrischen | Hacker News Reader

Can someone explain how this works? Do they use the GPS + compass to detect what you're looking at or do they actually analyze the image data. I'm thinking it's the former.

View on HN

Probably GPS + Compass + Accelerometer. You need to be able to detect the angle of the phone as well (looking up, level, or down).

The iPhone doesn't have the processing power to do meaningful real-time image recognition.

I would agree with you - but the 'infobox' (for lack of a better term) stays stationary over the same point of the building for the whole thing. I can't imagine the computations to keep it that centered based simply on the orientation of the phone would be either accurate enough or much faster than basic block/rectangle image processing.

After all, Google street-view does some simple image processing like that in-browser using JavaScript, and I'd imagine native iPhone code performance would approach that level of power.

With the accelerometer you can detect gravity (true down) and your angle relative to it. Then model the earth as a plane and keep the label a constant distance above the ground.

Seems like they did a good job of it though. It's really slick.

If they were doing feature tracking (following recognizable points in the video), this would far smoother. They don't even appear to be filtering, as their info boxes are shaking all over. Anyone demoing an AR app who moves the camera slowly is compensating for poor implementation.

GPS for location, compass for direction. And it'll only work reliably outdoors since there's no direct access to the camera stream.

Reply on news.ycombinator.com