Why we're all-in on virtual reality

Sometimes people build technologies that just aren't that attractive to people. I stretched a little for the Steve Jobs joke, but it turns out that sometimes you build something, it works, it's even pretty awesome, but a few years later it turns out that smartphones have 6" screens and tablets just seem less useful than they used to.

Or sometimes it's worse, and you build Google Wave to revolutionized and unify email, chat, and discussion boards and it turns out that people Just Don't Want your tech.

Events though I can definitely see spending real time inside of a headset just to watch video. Cinema too. Lots of studios (including Oculus's own Story Studio) are getting in on that already.

3D scanning is best for static objects like walls in your room. They never move or change, so scan the whole space. When you turn your head in VR, you'll see accurate 3D representations of that 3D environment.

Full 6DOF tracking is another method. Given a 3D scanned static object, like a coffee cup, you can track it using various optical and IMU-based techniques. This means a static object can be dynamically tracked as it moves, giving you accurate information about the 3D world to pick up your coffee cup without spilling it.

Optical tracking can be used to replicate facial movements into 3D models (see: Faceshift), to track finger movements (see: LeapMotion/Kinect), or full body motion.

Video can fill in the gaps for dynamic content. This can be achieved in two ways: 1) depth cameras which combine RGB video like a normal camera with depth information, allow us to reconstruct dynamic objects at proper depth. 2) 360-degree stereoscopic video which gives a fixed perspective to the full 3D environment. This isn't great for "3D from all angles" as non-rotational head movements won't see an accurate view.

A combination of these technologies will enable a VR user to see a real 3D environment, and feel like they are in it.

One thing is certain: it's coming. It's just a matter of time. The technology is rapidly maturing.

"Recap: the rig takes two Canon DSLRs and spins them around in a circle, capturing a dense light field image. The resulting data, when properly processed, can be rendered with OTOY’s pipeline and delivered to a VR headset with full head tracking. It’s a static scene but OTOY have alluded to future video capture."

They're not.

Think of them as a spectrum. VR + stereoscopic image passthrough gets you an AR-like view of the real world around you. These overlays allow you to pick up coffee cups, see when your kid is poking you, etc.

https://news.ycombinator.com/item?id=9938332

Valve is solving walking into walls by showing you a holodeck-like barrier inside the headset, so something like that could work with moving objects like kids or the dog over time too. Still, you are quite disconnected from the world that's actually around you.

In terms of making technology more human, I see VR as a stepping stone to Augmented Reality, which will eventually allow us to interact with our hands and instead of being in another world, you'll see things brought into your own world. That is an improvement over a screen too.

But VR can also transport you to places you've never been through 360 video, which is a powerful opportunity to create empathy through journalism and documentaries that can help us better understand how people live halfway across the world. I'm rather excited for that.

While any technology has the potential to further divide or distract us, VR has made me feel things that no other technology has even come close to feeling.

But if you look 5-10 years out, I wouldn't be entirely surprised to see drastically miniaturized components, perhaps a headband of sorts, a Glass-like eyepiece with a projector, maybe contact lenses, or something that provides a strong level of immersion without hiding the face.

In terms of recording the space, the Kinect is able to determine an astonishing amount about a physical space that I imagine would go hand in hand with this.

VR won't solve this in an elegant way immediately, although virtual avatars will soon have eye and facial expression tracking, so at least the avatar will mimic your actual expressions inside of a virtual space. The value add there is that you can be in a shared collaborative space with additional objects besides just other people, and interact with those together.

Companies like Lytro have started working on solutions to let you move your head around a little bit, but that's about the extend of video for the foreseeable future.

Still, you don't always want to be moving around in a scene. People sit and watch movies all the time, so there's going to be a place for it.