Hacker News Reader

You bring up a good point that covers why I'm most excited for the AVP: I'm really excited about what later versions will be like. This first AVP is kind of like the first iPhone for me. Obviously not perfect, but the UI will probably be far better than anything before it, and later versions are going to be really awesome.

It's really just math. You can't simulate a 3840 x 2160 screen with a 1920 x 1920 headset. So until the headsets get more resolution than monitors the monitors will look better. Especially for text.

You don't look at all of a 3840x2160 screen, with a need for pixel-precision all over it.

Doing a test right now, I'm focusing on the 13-inch computer screen where I'm typing this and holding up a large container of cashews directly to the right of the display. It's literally less than 30 degrees to the right of my focus, and the word "cashews" is in half-inch-high letters, and I can't remotely read the word without moving my eyes over to it.

> I can't remotely read the word without moving my eyes over to it

Plus the portion of your eye that sees in high resolution is ridiculously small—1% of your field of view. And your eye and brain just hide that fact by making your eye constantly jitter to see a larger area.

But, they aren't doing fovea tracking and shifting the display around in response to eye movements, are they? If not, you would only get more screen space if you articulate your neck and cause the head-tracking to update your viewing perspective.

I've monitored how I use 4K screens in the 28" size range. I would not normally move my head at all to look at different parts of the screen. I would only move my eyes. With two such screens side-by-side, I would turn my head just a little bit to focus on one screen or the other. Or, I found I would often hold one head position that is neutral between the two, and then use approximately 2/3 of each screen, with the far outer edges neglected or used to banish less relevant communications and status apps that I only check once in a while.

And, these screens are not filling my field of view by any means. So, I'd really need a far higher resolution headset if it is going to give me a good immersive field of view and be a reasonable monitor-replacement. I fear that fovea-tracking will remain scifi dreams in my lifetime, so the reality is we need to render full resolution throughout the field of view to be prepared for where the eye gaze might go in the next frame.

> I fear that fovea-tracking will remain scifi dreams in my lifetime, so the reality is we need to render full resolution throughout the field of view to be prepared for where the eye gaze might go in the next frame.

This is not at all true. All of the AVP, the Quest Pro, and the PSVR2 do eye tracking based foveated rending. They lower the clarity of the things you're not looking at. And reviewers say it works perfectly, like magic. They are unable to "catch" the screen adjusting for what they're looking at.

Hmm, interesting...

Are they actually doing some kind of optical shifting of a limited pixel display? Or you just mean they do some kind of low-res rendering mode to populate most of the framebuffer outside a smaller zone where they do full quality rendering?

In other words, are they just allocating rendering compute capacity or actual pixel storage and emission based on foveal tracking?

> Or you just mean they do some kind of low-res rendering mode to populate most of the framebuffer outside a smaller zone where they do full quality rendering?

This exactly. They don't reduce the resolution too much, but it's visible to an outside observer watching on a monitor.

That’s just to be about to have enough compute/bandwidth to drive the display. Other posters are correct that the DPI decreases away from the center and various optical aberrations increase. Foveated rendering won’t help with that.

Fovea-jitter is far from covering the width of your vision. They can overshoot by (maybe, I'm guessing) 2-3x the are of your precise vision and all jitter will be covered.

This isn't how any of this works. It doesn't matter how little of an area of the monitor your focusing on. You still need to generate an equivalent DPI over that area in the headset to match the monitor. Which you can't do since the headset has fewer physical pixels than the monitor does.

No you don't. The headset uses eye-tracking to enable foveated rendering. You only need to generate a high-dpi segment of the screen, where the eyes are actually focussed

Ok. Now let's wait for a display panel which can fluidly move physical pixels around to get the most of them where the user is looking!

You can't really "generate" a high-DPI segment.

The display either has a sufficient pixel density or it doesn't. Foveated rendering let's you cheat the details you render outside the focal point, but since your eyes can move to any portion of the display, the entire display needs to have a high-enough DPI that, once your eye moves to focus on that section, the content rendered in that location will be sharp.

The limit on DPI is from the display system. Foveated rendering doesn't help.

3840 x 2160 resolution per eye. 23 million pixels per eye, or, nearly 3x the pixels of a 4k TV with the equivalent resolution. That's roughly 3 physical sub-pixels per 1x1 "virtual pixel".

As well, given the oddities of human vision and how eyes focus, they can give more subpixels to the tiny area your eye is looking at to increase fidelity

The Vision Pro has 23 million pixels total, or 11.5 million per eye, or 3,391 x 3,391 if square

>That's roughly 3 physical sub-pixels per 1x1 "virtual pixel".

That is apples to oranges. The virtual pixels are mapped over your entire field of view where the physical pixels are mapped directly to the 2D surface itself.

>they can give more subpixels to the tiny area your eye is looking at to increase fidelity

Again this is a property of the display system and can not be improved by software. Additionally only one part of the lens (usually near the center) will have the highest PPD. Looking anywhere but the center will have a lower PPD.

It does when your display has micro motion :)

I get that I can look at the upper-right corner of the vision-pro's display and then the precision is needed there -- or the lower-left, etc. But no one is going to do that for long; it's uncomfortable, and you're going to turn your head so your eyes can relax.

I am pretty sure the Vision pro's resolution is higher than 1920x1920? Could be wrong through.

Regardless, the question isn't if it's better or the same. The question is if its viable considering the author is saying the idea is "Ridiculous"

Yes it’s higher. From what we know (11.5 megapixels per eye), the width roughly corresponds to a 4K resolution. However, due to the 3D resampling, I would estimate that to simulate a virtual monitor of a given resolution in high quality, you need at least 3-4 times the resolution on the physical headset display. Which would mean that the AVP couldn’t simulate even just an FHD virtual monitor in high quality.

I haven't seen official numbers. My point is that it needs higher resolution than a monitor to match the experience.

3840 x 2160 resolution per eye. 23 million pixels per eye, or, nearly 3x the pixels of a 4k TV with the equivalent resolution. That's roughly 3 physical sub-pixels per 1x1 "virtual pixel"

> 3840 x 2160 resolution per eye. 23 million pixels per eye, or, nearly 3x the pixels of a 4k TV with the equivalent resolution

So a 4k TV is 3840x2160. The equivalent resolution is... 3840x2160. The number of pixels in 3840x2160 is 1x the number of pixels in 3840x2160, not 3x.

The Vision Pro has 23 million pixels total, or 11.5 million per eye, or 3,391 x 3,391 if square

Is it _trying_ to simulate a 4k display? Or is it trying to provide something different- a large panorama of 1920x1920 windows?

Neither really. The VP uses a pair of 4K displays to render a high dpi section where it sense your eyes focusing. This is out of a much wider virtual display space (potentially 360deg). As your eyes and head move it shifts the image through that high resolution area. the space outside that area is rendered in a lower resolution to match your peripheral vision. This technique is called foveated rendering.

People who have tried the demos say that it is a seamless experience. Moving your eyes and head around, you see things rendered at a 4K resolution but it appears that the space around you is unbounded.

This is wrong. The eye tracking is only to reduce the computational demands, it doesn't change the DPI of the screen *at all". It's purely an optimization trick.

I don't know why people keep repeating this myth. It's just not how it works. It's not moving around pixels in the physical display.

>I find it kinda wild that we are seeing article after article of people dismissing this without actually experiencing it first hand.

Current VR has fundamental problems that can't all be solved with just some clever software tricks. This article is stating some problems that the headset will have to face. He doesn't dismiss the headset in the article.

>Once the hardware manages to catch up, is smaller, etc it feels kinda nieve to think that we would remain stuck to monitors for computing.

The friction of having to wear something on your head, along with the weight of the device will always make monitors a good option as they are very good at being 2D displays in the word.

>We saw a lot of the same dismissal of smart phones for most people's computing needs.

Show me where people dismissed smartphones over them having bad text rendering.

People dismissed smartphones and tablets because they have bad text input, which isn't exactly the same, but the purpose of their statement is not to say "the literally exact same criticism was leveled at phones" but rather "people also had somewhat similar objections and look where we are now in spite of that".

If you are being honest with yourself, you will remember that when the iPhone was first unveiled, you, I, and everyone that currently browses Hackernews thought the idea of a phone that was only a touchscreen and a few buttons was absurd. The iPad was similarly seen as ridiculous. We now live in a world where tablets, smartphones, laptops, and desktop computers all have their own niches and where a standalone desktop with a dedicated monitor, keyboard and mouse is seen as a specialist device. A headset may become yet another entrant in the "yes, and" world of computing. The historical experience suggests that we should have some humility around predicting what people will latch on to.

> I find it kinda wild that we are seeing article after article of people dismissing this without actually experiencing it first hand

I suppose you haven’t been following Apple News for long. The cycle is always the same, everyone is explaining why it’s a worthless product, too expensive, no market, doesn’t work, whatever. And then later they’re all about how obvious of a product it is, they or their favorite manufacturer did it ten times better, Apple needs to give the technology away.

Remember Slashdots out-of-hand dismissal for the iPod: ‘No wireless? Less space than a Nomad? Lame’. It sounds so obvious why the product is bad, but it turns out it’s focusing on things that don’t matter.

> it feels kinda nieve to think that we would remain stuck to monitors for computing.

One simple reason: “Hey Stephen, is this what you were looking for” (points to picture of x device on the monitor) Stephen- (without even raising from his chair) Yeah but in red, where did you find it?

Whenever I read about the AVP being a dud, I am reminded of my own reaction to the first iphone: “it is a stupid idea and it will never catch on” whilst proudly clutching my Nokia communicator.

I think the AVP 2 or 3 is going to be great.

The article IMHO does a great job breaking down the pixel arrangement, troubles with rasterization, how modern monitors basically cheat to get sharper text, and so on. TL;DR: VR/AR suffers greatly with text.

The only way Apple will potentially mitigate this will be via clever rendering tricks. But he is very likely correct: this will not be a replacement for monitors.

So, one problem with this statement: People are working with virtual monitors today using far worse DPI equipment.

This is an iteration on something that already exists. Probably won't be perfect, but it can and will be a monitor replacement for some.

It's not far worse. My display at home has 60% of the Li ear resolution of the AVP, the experience is terrible for productivity. I've tried displays that come very close at 80%, and it's still unusable for me. If the screens are reasonably size so that I don't have to move my neck all the time or suffer optical aberrations, like my real life screens, I get an effective resolution of ~720p.

We can see that this will be similar : the AVP has ~35 pixels per degree, my screens are about 45 degrees, so I can expect 1575 pixels across and 885 vertically, so just a tad better than 720p. In practice you'll lose significant resolution to optical aberrations unless your neck makes it point right there.

Who is using 24inch 720p displays in 2023? You can get at least a 1080p for like 20$ on classifieds and 100$ new.

This is sitting as close to the display as you ever should, btw. A 24inch screen will be 45 degrees wide if you're sitting less than 1 meter (36in) away from it.

> it appears they are using a virtual screen that is far larger than what most people likely have. meaning the text will be larger

I mean it's just a concept marketing photo - the screen doesn't actually project out to be seen by anyone else. The projection is made for friggin' ants (kinda literally).

I realize that it doesn't project out, but for every reason that matters it does for the person that is wearing the headset.

https://i0.wp.com/kguttag.com/wp-content/uploads/2023/06/AVP... look at that image. Judging by the guy in front that middle virtual monitor is probably 3 times the size of my current monitor which is 32". If I want more than 1 monitor (I have 3) I can't imagine actually having anything larger than that in a physical space. But in virtual? Why not. And that is what apple is showing with these images.

I feel like neck stress/pain will be a thing in VR/AR just as it is in real life, so monitor size and positioning will have to follow at least kind of similar principles as IRL ones do if you care about ergonomics at all.

One scenario I'm hoping for: I had a device that was about the size of a lighter (but even smaller) that you would wear around a necklace under your shirt, between your shoulder blades. You'd calibrate it and over time it would train your posture as you slumped by vibrating gently.

I'm not trying to comment/ debate on if this will ever be a mass-market product, but, it would be fascinating if they built a posture corrector into it, and it ends up that AVP users develop better posture compared to keyboard/ mouse/ screen users

That is a valid point, and I am curious how that part will work out in the real world.

I do think about my setup, how much space is basically lost to speakers, space for my keyboard and hands (not that I wouldn't use that in this setup but I could kinda not think about it). Or just non traditional aspect ratios as shown in that image.

But I agree, I think it remains to be seen how that works in the real world as far as proper sizing and ergonomics goes. But I think there is room for some expansion on what we would normally have now.

Again, it's just a concept photo. The graphics designer hasn't used the product, nor has the model. They made the image to look pretty - the model, the monitors: they're all laid out in the rule of thirds. We shouldn't try to tease out what it should be like using this thing.

To play devil's advocate, your 32" monitor is more than 3x as large as the 13" monitor I'm working. People use interfaces on wall-mounted TVs that are 3x your 32" monitor just fine.