Comment by moron4hire | Hacker News Reader

Comment by moron4hire | Hacker News Reader

Parent Full thread

> no developer is paying attention to proprioception

That is not true at all. Proprioception is top of mind for every serious VR developer. Outside of poor software performance, it's the #1 cause of simulator sickness. Short of hacking the vestibular system, there's nothing we can do about the lack of virtual acceleration cues. So we design within the constraints we have. It's why you see teleport locomotion. It's why developers get into such arguments over "smooth movement" with gamers who think they know better. Lack of proprioception with virtual movement is a huge problem that we would love to solve, but laypeople have an irrational fear of a little extra electricity getting piped through their ears, so I doubt we're going to see much change there.

Fixed-view, wearable displays violate proprioception even more than VR HMDs, because they don't even give you the real acceleration cues you get from actually moving your head.

As for your question about distance, I suspect you're looking for "accommodation". Accommodation is the physical distortion of the eye to change the focal distance of the lens. Combined with vergence (the distance at which our binocular vision rays converge), the two systems work together to give you a sense of how far away something is (among other things, like atmospheric hazing, parallax, and trained memory of similar objects). Most modern VR systems lack the ability to account for accommodation, leading to what is called "vergence-accommodation conflict". It can be a source of eye-strain for some people, but it tends to be adaptable over time.

Note that I said "most" modern VR systems. It's an active area of hardware research, as the solutions--light field displays and varifocal lenses--are large, complex, and therefore expensive. The latest developments in "pancake" optics, where focal distance for the lens is folded in on itself via a clever system of internal reflectance, has a possibility of bringing the size down, but unfortunately not the complexity or cost.

So again, it's really not fair to say developers are "ignoring" these problems. We are all extremely aware of the them. There are just some things you can't do anything about without a major breakthrough.

> Fixed-view, wearable displays violate proprioception even more than VR HMDs, because they don't even give you the real acceleration cues you get from actually moving your head.

This is unfortunate, I have severe motion sickness; Can't even play Minecraft or any FPS. So VR headsets are out of question.

I was waiting for display glasses like Lenovo's T1 for years now[1] due to its accessibility benefits. From what you say, It seems like I would be sick if I use display glasses too until the proprioception issues are resolved and I'm glad that it's being seriously researched.

[1] https://needgap.com/problems/16-wearable-low-latency-display... (Disclosure: My problem validation forum).

Are you saying that you have tried VR headsets and find that they are bad for your motion sickness, or that you presume that they would be bad because of FPS games?

If it's the latter, there is a small chance that you would not have the same motion sickness problems with a VR HMD as you do with FPS gaming. Most of the headsets on the market today use 6-degree-of-freedom tracking for the head and hands. So if you move your head, the view does update accordingly. Actually, I'm not even aware of any headsets that are still available for purchase that have 3-DOF head tracking, though there are a few like the Vive Flow that have only 3-DOF controllers.

Headsets like the Quest 2 and Valve Index also have extremely high screen refresh rates (120hz and 144hz, respectively) that have practically eliminated for most people most of the motion sickness problems that can be attributed to the hardware alone. Granted, there are still some dodgy apps out there. I would avoid anything based on 360 video content in your case. Hell, I don't really have any motion sickness problems and I avoid 360 video content, just because I find non-interactive stuff to be a poor experience overall.

I wouldn't go out and buy one right away, but if you run into someone with a Quest 2, I'd encourage you to try it out. Supposedly, Meta is about to release their next iteration of headsets, whatever "Cambria" or "Quest 2 Pro" or "Quest 3" are supposed to be, which should be even more powerful.

The point I was trying to make in my post was that GP didn't know what he was talking about and that devices like the Lenovo T1 would be a lot worse for the kinds of issues he was mentioning than the VR headsets he was complaining about.

I just presumed that VR would suck for me since I couldn't play any FPS and didn't try any real VR headset.

Thank you for the insights, It makes perfect sense for why VR headsets with greater refresh rates and 6-DOF might not make me sick. I'm going to try it when I get a chance.

> there's nothing we can do about the lack of virtual acceleration cues.

I think we're not talking about the same thing. I don't see how it could be a cause of simulator sickness, which I thought had to do with eye strain from focal points not being correct, and the inner ear.

Proprioception is a sense like the ordinary 5 senses, the perception or awareness of the position and movement of the body. It is what makes our body feel like our body. Dissociation, caused by dissociatives, can cause inanimate external objects to feel like they are part of us, which could be considered chemically hacking proprioception. The wooden arm illusion is another way to hack proprioception using touch to reinforce the sense of it. But VR should be able to do this without chemistry or touch, because visual clues reinforce the sense of proprioception.

VR can really be the wizbang killer app, but it isn't. Maybe they show you a virtual arm and you see yourself manipulate it, but it doesn't feel like your arm (I am not talking about the feeling of touch). That is how they are ignoring proprioception. And it doesn't have to be an arm, because in VR, you don't have to be human. It should be possible to blow the wooden arm illusion out of the water with VR, and have ten arms, and they all feel like they are distinctly your arms. But we don't see that, and no one is working on it. All we get is a stiff arm without natural reticulation.

So it is fair to say developers are ignoring proprioception, because they are unaware of their own sense of proprioception (like knowing where your hand is without looking at it). Proprioception should be of primary concern in Virtual Reality. They should toss everything, and start all over again with inducing proprioception as the prime driver of the technology.

> As for your question about distance, I suspect you're looking for "accommodation". Accommodation is the physical distortion of the eye to change the focal distance of the lens. Combined with vergence (the distance at which our binocular vision rays converge), the two systems work together to give you a sense of how far away something is (among other things, like atmospheric hazing, parallax, and trained memory of similar objects). Most modern VR systems lack the ability to account for accommodation, leading to what is called "vergence-accommodation conflict". It can be a source of eye-strain for some people, but it tends to be adaptable over time.

Interesting, but complicated. Should be called "apparent display distance," or something simple and easy to understand. Still, gives me more information. Appreciated.

A non-trivial number of VR users experience phantom sense of varying strength, including myself. Proprioception is not the only thing being consulted here, seeing someone touch what visually seems like your face is a common low-level manifestation of this. It's possible to be in an avatar and have someone touch the avatar's tail and somehow weirdly you can feel like "your tail" is being touched (albeit you feel it in your tail bone).

Maybe the technology to better integrate more brain variants will eventually exist, but current VR tech already has a niche to cater to and develop from.

> A non-trivial number of VR users experience phantom sense of varying strength, including myself.

You have elegantly spotlighted the problem. Call me when they all experience it, always and every excursion into the virtual. Then we'll know VR developers have finally started paying attention to proprioception.

Even this, what you are saying about "virtual arms", has been tried. Inverse-kinematic self-avatars were a big thing within the industry about 5 years ago. And it was... largely not that important. Without a tracking point for the elbow, it's nearly impossible to do IK for limbs. Remember, a double-pendulum is a chaotic system. A small perturbance can lead to dramatic divergence between simulated and actual state in chaotic systems.

That said, it's still possible to get body sense in VR, for the head, main torso, arms, and especially hands. It's hard to do. And when things are hard like this, there are economics of scale to consider. For example, the Quest 2 has native hand tracking built in, but not a lot of other VR systems do, and the ones that do often have dramatically different APIs (though OpenXR helps a lot here, which shifts the problem to whether or not the system in question has an OpenXR-compliant driver). Do you go all in on one platform and support its input modalities to the maximum degree? Or do you try to target the common denominator between a broader swath of the market.

Non-VR consumer software developers get to develop their software in an environment of operating systems that have well-established UI and UX metaphors. You make a button in WxWidgets or UWP or SwiftUI or HTML and that button comes with a LOT of functionality that you don't have to program yourself. We don't have that in VR. To make a button in a game engine for a VR game you basically have to implement that entire thing yourself. Oh, the game engine probably gives you a raycaster to be able to find objects, but you have to program up where that raycaster should be firing, which objects you should be interested in, and what interactions hitting the buttons on your controller map to how the button in the view reacts. To a large degree, every VR developer has to reinvent the world when they make something. We're working on the very edge of performance that these systems can eke out.

We don't have shared UI metaphors, but not for lack of trying. There are several projects underway attempting to do just that. MRTK and StereoKit in particular have put a lot of work into just that. But they aren't for everyone. They can't do everything. It'd be like expecting Call of Duty to be implemented strictly in Win32 controls. And the input modalities are so much more diverse and non-standardized than desktop mouse-and-keyboard that I doubt there will ever be a one-size-fits-all UI toolkit for VR.

I reassert that you are not being fair to the VR industry. Everything you have talked about so far has been tried and either found to not be worth the complexity or not possible to do within the current constraints.

> it's still possible to get body sense in VR,

Accidentally stumbling upon proprioception is not what we're after. We are after being fully proprioceptively immersed within seconds of jacking in, and until developers place proprioception front and center in their development of VR, accidentally stumbling upon it is all we can expect.

I don't have enough prior knowledge that I can't tell your definition of proprioception is correct or not, but I know that forward kinematics is completely neglected and that should at least be included in your version of it. Even teleop robots weirdly don't have full limb/joint angle capture nor cares about forward/inverse kinematics agreement, just inverse with control handles.

Reply on news.ycombinator.com