I think that in the future, the following phenomena are likely to proliferate:
- high-res satellite and airplane imaging of the globe
- public geotagged photos, including compass bearing
- Google Street View road images
- public security cameras on public streets
All of this data can be integrated using software like Microsoft's PhotoSynth, to stitch together a gigantic 3D model of the real globe.
Then, people with Voxel Vision will be able to look around at streets and landmarks that are interesting but physically occluded.
======
Edited to rise a relevant comment from further down the thread. Original answer follows:
Jon Rodriguez here, project lead.
Your question seems ambiguously worded.
I will answer both possible interpretations of the question:
===
Q:
"How do you propose to do the sensing task of seeing inside real world objects so that that data can be rendered?"
A:
The source of content depends on the application.
1: put a virtual screen at distance infinity, so in a boring meeting you just look far away to check your email
2: for telepresence surgery, use a MRI to do the 3D imaging of the patient's interior
3: Google Street View images allow you to see to any point in a city, even if buildings are between you and it
4: educators create custom 3D models of physical phenomena they want the kids to vividly explore, such as a 3D model of a human neuron, or a 3D geological model of the Earth, or the interior of a grape.
Etc.
In particular, virtual reality games and worlds can be easily shown this way, as long as the game artists take the extra effort to give their models interesting interiors.
===
Q:
"How does a Heads Up Display that is in front of a real-world object (like a brick wall) then project a virtual image that appears to be behind that object?"
A:
This is where the adjustable lens on the virtual retinal display comes in.
By properly decollimating the light from the VRD's screen or laser, you can give that light an arbitrary "equivalent optical distance".
Despite the fact that the light is originating at 2 cm from the eye, it can appear to originate at 2 km from the eye, in that 2 km is that distance that the eye will have to accommodate to to put that light in focus.