Hands-on with HoloLens: On the cusp of a revolution
arstechnica.com
arstechnica.com
I shook my head back and forth like a dog trying to dry itself off, and the images I saw barely wiggled. The images are bright enough to completely occlude reality. The software dev stuff is from MS, so it is polished, simple, and powerful.
Voice recognition on the device, for example, is just handled by calling the voice functions in the core library, then GIVING IT A STRING. It listens for this plain text string. That's it. Sooooo simple and powerful.
Anyway, I see downthread that VonGuard has made the silly claim that "Hololens can do everything Magic Leap is supposed to do", so I think it's safe to say that the "wasting money" thing is just a throwaway line that's not based on a lot of actual knowledge in the field. I feel kind of trolled.
It's a VERY good device. Surprisingly good. I expected it to suck.
Hololens is just a screen in front of your face. It could never blind you or hurt your eyes. It just works, it works today, it fits in a product form factor today, and after using it for a while, I really don't see much need for an also-ran here. Hololens can do everything Magic Leap is supposed to do, and it can do it a LOT cheaper and safer. Plus, MS dev tools. You just know that Google's Magic Leap dev kit will, basically, be targeted at super intelligent people and require a lot of do it yourself stuff.
See:
https://en.wikipedia.org/wiki/Liquid_breathing
I really can't believe that the light level in MagicLeap isn't adjustable to a comfortable level.
https://en.wikipedia.org/wiki/Virtual_retinal_display
http://www.hitl.washington.edu/research/vrd/
http://ascent.atos.net/ascent-look/
And here's a (terrifying) DIY version: http://eclecti.cc/hardware/blinded-by-the-light-diy-retinal-...
Some tech comes out early but doesn't catch on for a reason.
You can interact with others, your environment, and other systems without the isolation or nausea of VR. On top of that, being able to supplement the real world instead of only temporarily replacing it seems far more valuable to me.
Now we just gotta shrink it down to contact lens size, and we're good to go!
edited for formatting
And VR doesn't cause nausea. Bad applications cause nausea.
So fundamentally VR has more potential than AR. Things like nausea may take some time to work out but they're not unsolvable. For example, if the issue is the disconnect between the acceleration/orientation sensors in your ears and the virtual reality we can just bypass those sensors and inject a signal directly into the relevant nerves. It might even be possible to do this through the skin in combination with training. The frame rate/resolution issues will also improve until eventually virtual reality will be indistinguishable from reality (ignoring tactile input for a sec). Sure physical movement in this environment is another problem. But the end game is something that is simply incredible/unimaginable.
With AR- sure you can add more synthetic information over reality. You still have tactile challenges (you can't touch it or interact with it). It is very difficult to combine seamlessly with the outside environment to make it more than just a fancy heads-up-display.
Not saying AR isn't cool and full of potential but I don't think we can compare its potential with VR. VR is the holodeck... AR is ... a heads-up display? Not to mention the need to carry AR with you wherever you go if you want to use it in the "real world", power it, compute, etc.
EDIT: One more thought while I'm at it. You can always inject the real reality into VR and thus make it AR but you can't remove reality from AR... Or if you could the two would basically merge. So AR can be seen as VR with "transparency" ... And VR is a superset of all ARs where you can't completely remove reality.
Fundamentally, VR just has a different potential than AR.
> Not to mention the need to carry AR with you wherever you go if you want to use it in the "real world", power it, compute, etc.
You need to carry the VR headsets and their host-devices with you too.
I believe we should clearly delineate the definitions of VR vs AR first.
Virtual Reality should be things like The Matrix. A reality that an individual experiences without modifying the real world for anybody else. Current technology simply shoves a screen right up against our eyes but the goal is obviously to plug that content straight into our very brains.
Augmented Reality should be things like an audio speaker or a video display; devices that "inject" manmade media into the real world; devices which generate reproducible experiences that everybody, even animals, can see, hear, touch and smell.
In AR a single device may provide content passively to multiple viewers. Like your music system or television or a hologram generator. Even your pets can see and hear whatever's playing and they certainly don't to purchase or wear anything.
VR aims to interface with your neurons and you're guaranteed to experience the full content. AR projects content into the real world which may or may not bounce on to our senses.
TL;DR: VR replaces reality. AR enhances reality. But that's just my opinion. :)
And to maybe extend the discussion abit, what about devices that plug right into the centers of consciousness. Is that just more invasive, arguably better AR or is it VR? What do better hearing aids that sample at 10 ghz and connect directly to the nerves instead of hair cells. What are they considered? What about when I pipe my FLAC uncompressed music through the hearing implant -- VR or AR? It seems like the future is a combination of the two in far more complex ratios than just one or the other or split screen. VR probably has limits of perceptiveness if it fails to take model of logical non chaotic reality. It therefore can eventually become an abstraction of an existing event or well..present stimulus. Anyhow, I think the ideas are all really profundo. Looking forward to replies.
[1] https://teddybrain.wordpress.com/2013/08/28/a-brief-review-o...
TV can be considered a primitive form of reality augmentation. As I said even your cat can see and hear what's on. Even aliens who see in an entirely different spectrum (or don't have any vision at all) will be able to pick up the colors and sounds generated by that TV, through their monitoring instruments (just as we can "see" infrared and detect magnetism etc. through our tech.) A television set modifies reality in the space occupied by that set, for everyone, however minor that modification may be. By that definition I suppose even paintings can be considered AR.
Again this is just my opinion. I don't know how the Star Trek holodeck exactly works (in-universe) but it would be AR, as I'm assuming that anyone outside the holodeck will be able to see the generated visuals if they were to peek in through a hole, as it were.
Or looking at it another way, isn't VR just AR with the lights off?
See, that definition is much much technically simpler.
I couldn't agree more completely. I live in a physical world, and my god I wish I could connect it better to the information world.
I don't care at all about VR. I'll try it because it's new, but I have close to zero interest in gaming and beyond that there a very few use cases where it is interesting. Design maybe, but a lot of these uses cases actually work better as AR anyway.
You argue that we can bypass the orientation senses by injecting signal into nerves, but that the tactile challenge in VR is impossible?
Yes, you have to wear it. But glasses aren't a big deal - and will keep getting better - and AFAIK you have to wear VR gear too.
Mmmh... you say you agree with the previous poster but your message seems to disagree completely.
So AR is the superset of VR, where a physical cover makes them identical when it is on.
The gaphics integration into the real world is phenomenal. If you put a virtual object in a room you can walk around it. If you put something on a wall it stays there no matter where you walk.
Unfortunately, the interface -- how you interact with virtual objects -- is completely janky. Awkward pinching gestures to select keys from a floating virtual keyboard. Cursors that are supposed to represent your physical movement, but instead jump around.
It is tantalizingly close, but it seems like it would be just as annoying as it is neat or useful until someone comes up with a better (simple and reliable) interface.
Another interesting depth issue is when it ran to the back of the table and straight on to the floor as if they were continuous, because the view angle made it so the table and floor were right next to each other only separated by a line, so the fox didn't do a jump animation. I suspect a depth map of the room needs to be constructed to solve that issue, no mean feat.
These kind of issues just show me this tech is still years away from being what we'd expect.
That, plus the collaboration features would make HoloLens pretty awesome for any kind of collaborative product design. Take cars, for example. Right now, car mockups are done in clay, and while clay is easier to mold than metal, it still takes time and limits your ability to iterate. But with Hololens, you can have a design review where each of the participants puts on an AR unit, and now you can all see the same car and make changes to it in real time.
I'm excited to see what sorts of applications will be opened up by being able to display virtual objects in a "real" setting. I also agree that this won't be consumer-facing technology (at first). This is the sort of thing that'll take off with architects and engineers before it takes off with "normal people".
Pure speculation, but I would bet that it generates the room geometry at startup and then doesn't change it later. The device has limited processing power and needs to hit a high frame rate, so doing that makes sense.
A person walking into the scene wouldn't be accounted for in that case. It's also dependent on scans of the environment and the quality of geometry reconstruction, so things like a gap between objects that you didn't see from a good angle could easily run into that issue.
The depth maps can get dynamically updated, so it's not insurmountable, but currently the process seems to take a few seconds. I'm not sure if this is a design decision, a processing limitation, a hardware data capture limitation, or something else entirely.
Hot Dang! I guess that ought to keep a VR object stable.
1. It's version 1.
2. It's an interface that nobody's used before.
Neither of these is a mark against the technology.
Go ask a caveman to use a version 1 iPhone, and see if he falls in love with it or just trash-talks it on an internet forum.
That being said, maybe the hololens can be good enough nonetheless, because interaction latency is separate from pose/presentation latency and the latter is certainly far more make-or-break.
I suspect that the main achievement (or magic sauce) of the hololens is strict separation between position-finding plus persective-correct content rendering (which both together are quite high-latency, relatively speaking) on one hand and the final push to the actual framebuffer on the other, where a last corrective shift(/and maybe rotate(/and even more maybe zoom)) correction can be applied independently. The latter would be very low latency, based on the difference between camera frame at the beginning of the rendering process and most recent camera frame. This would be closely related (if not derived from) software-based shake reduction in cameras, which in turn might be traced back to the algorithms used in optical mice (which lack rotation/zoom).
I'm not sure if position finding for the head is very separate from rendering, just because it is so smooth when he moves. I think head positioning is very separate from gesture positioning -- which is why cursors are jumpy. Maybe 13of40 is right on getting used to the interface -- maybe the user is holding his hand out too straight, not straight enough or something to that effect, but that still seems more of an interface problem than user problem. I do wish it was priced closer to oculus. At that oculus pricing I would probably get one, but at 4x it seems not quite there (interface and lacking apps).
13of40 - I think it is more "version 1" issues with the shake and a lack of the best human-to-computer interface rather than an "interface that nobody's used".
Put-that-there[0] is an interface from the early 1980's that seems like it would do well to reduce the janky interactions. Speech recognition probably isn't quite where it needs to be for a generic setup (as opposed to the structured setup of put-that-there).
Like I said, the computer to human side -- rendering virtual objects over a real field -- is beautiful. The quirks with incomplete map information aren't too distracting because we are good at filling in the gaps. The human to computer side needs work because it takes 30s to do a simple resizing operation. If you could say "make that screen 90 percent of my field of view" at any time the vid author wouldn't be complaining that getting too close causes clipping of his field of view.
It's the standard structured IR light over the scene. Which means that sunlight, reflected sunlight, florescent lights, or any other IR emitter will wreck havoc over the SLAM algo they have running.
What they need is a visual SLAM and not just the IR solution. It works well in their dark candle lit studio where they did the demos. But outside, near windows, near reflections from outside, and flood sources all kill or badly affect it.
Do you know if the structured light is made by the unit, or is it an unmentioned peripheral?
It's all driven by the unit, btw; no need for markers or external illumination.
I'm not saying this would remove the use cases of AR, just that it could be a significant enhancement or enabler to many VR experiences.
The picture stuck to the window almost perfectly, which as someone in VFX I find very impressive.
Tracking head orientation and location is possibly the hardest thing to get right(next to object recognition)
Next ten years will be spend with feeding everything to it. He who wraps the last layer around the world, to sell it to the users, makes everything wrapped his organelles over time. Well played Microsoft, well played.
If we could decentralize the way consensed augmented reality is shared, we could even do something good for the world. Open Source WiFi Hubs offering free processing power and untraceable sharing. Mmmh..
[edit: grammar]
I had a Vodafone R&D team working with the Sony 3D AR glasses (aka 'Tom Hanks CES glasses'). We loved the tech and had a great playground, but couldn't get to the creation of one app that solved a clear painpoint.
We ended up with glorious tech demos, but they were all suited for a short-lived themepark attraction but not for ongoing use.
Anyway, I am excitedly waiting for someone who can do better than us.
Maybe IoT plus some connectivity / control standard similar to DLNA would make for a good base, where for example you could look at devices and bring up their menus to control them.
Real holography doesn't require glasses to view, and is not subject to the limitations and discomfort of route stereographic effects. This has been around in one form or another since the early 90's, it never gained wide appeal because of the fact you are secluded from your environment by the headset, and because using it (due to the inherent nature of stereographic effect with fake, nauseating parallax) can only be done for short time periods.
History repeats itself, and life imitates art.
Edit: For more on why I think this is a huge fad, just read this article about the death of 3D TV (which is fundamentally based on the same stereoscopic concepts):
https://www.avforums.com/article/in-memoriam-the-death-of-3d...
Choice quote:
"However the single biggest obstacle that 3D faced wasn't different versions, incompatible glasses, exclusivity, lack of content or screen sizes, it was simply that people didn't like wearing the glasses at home. Consumers were happy to wear 3D glasses at the cinema, which are predominantly the cheap and light passive variety but they were less keen to do so in their lounge."
Consumers didn't like wearing glasses then, and they won't like wearing them now.
Your only strong argument is that people don't like to wear glasses, but that could change if it is considered valuable enough. Have you seen the demos of what they can do?
And while certainly VR goggles may eventually share the same fate as 3D TV, I don't think the comparison is valid (other than they both involve stereoscopy).
It's a very different thing when sub-millimetre head tracking is involved, and when the virtual interocular distance is fixed to what a person's actual distance is. When everything appears to be the correct size and fixed in space, it crosses a significant perceptual threshold. I don't know if you've ever put on modern VR goggles, but there's no way I'd put it in the same category as 3D TV; in fact I don't think I'd put it in the same category as anything else I've experienced either.
And so I don't think you can take the lessons learned from people's willingness to wear glasses for a low-incremental-value 3D TV experience when the VR goggles have a markedly different offering - it's certainly plausable to me (though far from certain) that VR's value is high enough that people will be willing to put up with glasses.
Is it comparable to conventional screens?
Just wondering whether I can replace my monitor by this thing :)
A) almost no software right now is designed properly for VR/AR. The best we can do right now is splat a WIMP interface inside a cylinder centered on you. We haven't figured out or translated much of anything to a VR native interface yet.
B) the resolution-cross-FOV is just too low for current, standard WIMP applications on all current headsets. I think a VR native interface wouldn't have as many problems at the current low resolutions, because the depth gives you more to work with for reconstructing images in your brain as you move around the objects. But again, they are largely missing still.
We'll get there eventually, and I'm banking on sooner rather than later. But as of right now, you won't be using any HMD as a desktop multimonitor replacement for anything other than driving or flight sims.