I would agree with you - but the 'infobox' (for lack of a better term) stays stationary over the same point of the building for the whole thing. I can't imagine the computations to keep it that centered based simply on the orientation of the phone would be either accurate enough or much faster than basic block/rectangle image processing.

After all, Google street-view does some simple image processing like that in-browser using JavaScript, and I'd imagine native iPhone code performance would approach that level of power.