Apple Vision Pro – Potential hardware issues
kguttag.com
kguttag.com
The failure of 3D movies is always used as a cautionary tale, but it was a pretty unique case that came down to artistic execution. Both Avatar movies are incredible in 3D and don’t generate any kind of discomfort, but the average “3D” title from other studios was a cardboard-cutout mess, usually a worse experience than it’s 2D counterpart. Not a technology adoption problem.
Interesting. First time I've heard it mentioned that way.
My viewpoint at the time was that the 3d craze in tvs' was more due to manufacturers looking for the next big thing to compete with each other, and Avatar had come out recently and was doing well. Thus "let's add 3d" and every other kitchen sink / feature they could find. Thus smart tv's now, etc.
3D was one such feature, but had a benefit for non-3D movies in that it needed significantly more backlighting for active glasses to not give an overly dim experience.
3D was also odd in that normally such features get adopted first by a higher tier "early adopter" market - but because of Avatar and other family friendly 3D content, it was generally purchased by families. Perhaps its biggest early negative was expensive active glasses handed to young children.
HDR was another, but is somewhat spoiled by the lack of any good certifications/branding. Anything which can _process_ a HDR signal will be sold as a HDR television.
Curved was so weird to me, but that is because I always mount my televisions. Curved was a byproduct of the processes allowing for it. Since the curve has a fixed radius, it does lock you into a viewing distance 'range' that isn't always going to work for people.
4K was yet another, and partly what pushed 3D out in the US market. Active 3D also had a hard time with pixel ghosting - each frame was different from the previous.
Smart TVs are a different approach - sell advertisement and premium channels, I suspect in the future even put services like Uber Eats on TVs, all for secondary revenue channels to make up for how tight the primary sales channel is.
For example, you keep saying that a lot of theaters had to buy expensive projectors for Avatar. However, the article you linked to does not imply this. Sure, I imagine some theaters did buy digital projectors in time for the premiere. But all the article says is that many theaters didn’t have digital projectors even just 1 day before Avatar’s release. So no, most theaters weren’t buying digital projectors and 3D glasses specifically for Avatar. They couldn’t have. They bought them afterwards for all of the _expected _ future 3D movies. The retrospective article I mentioned above explicitly states this.
Also, the statistics used in your article are a bit misleading. Sure lots of cinemas around the world did not have digital projectors at the time, BUT the _big_ ones (that serve millions of consumers versus hundreds or thousands per year) did have digital projectors.
Also, for example, you keep saying that studios “needed to do something” with Cameron’s 3D technology since it was expensive to develop. However, this is not true at all. Only Cameron used his technology. Most of the films released after Avatar weren’t even shot in 3D. They were shot in 2D and then converted to 3D which led to poor viewing experiences.
Avatar’s influence on the 3D industry seems less direct than you’re making it seem.
Here’s the 2022 retrospective article titled “What James Cameron and ‘Avatar’ Did (and Didn’t Do) for 3D filmmaking”:
https://www.theringer.com/platform/amp/movies/2022/12/13/235...
There were 3D movies before Cameron was born. It’s been an ongoing cycle in movies, largely driven by the fact that the experience can be better, but it can also be neutral or worse, and the core of the art form is storytelling.
Cameron showed how to use 3D to support storytelling rather than distract, but these was no conspiracy to “recoup investments” — there were just imitators who didn’t put the work in to get it right, and manufacturers who were selling shovels and expected a gold rush that never appeared.
Our economy, and trends in the movie business are not centrally planned the way you imagine. Nobody says “AMC invested in 3D projection, so MGM should invest in 3D production, so this film that’s in development should be shot in 3D”. It doesn’t work that way.
For starters the 3D cinematography technology created specifically for Avatar was legitimately innovative and better than anything in the past (though still a terrible experience imo).
When Avatar was in production barely any cinemas had the projectors to screen it, and most of them had a big and expensive rush to upgrade for Avatar
> In the UK alone, only around 320 out of 3,600 cinemas are digitally equipped, while in the US the ratio is even worse (2,500 out of 38,000). "So there is a big problem looming," admits Peter Buckingham, head of distribution and exhibition at the UK Film Council. "You are looking at about a minimum of £80,000 to get yourself into a 3D position. Even with the hike in ticket prices and the potential hike in audiences, that's quite a stretch for the smaller venues. The danger is that, in this digital switchover, a number of cinemas may well be left behind."
https://theguardian.com/film/2009/aug/20/james-cameron-avata...
I vividly remember this at the time, and the huge amount of FOMO coverage of stories about whether your local cinema would be able to play 3D Avatar. Obviously none of this would have happened without the expectation that a wave of new 3D movies were about to be released.
The direct results of James Cameron wanting to make cool new 3D technology for Avatar was an immediate (and permanent) increase in ticket prices at the cinema, and 10 years of awful 3D movie screenings from cinema desperately trying to claw back the money they wasted on it.
Demand drives all things.
I dislike the power loss failure mode - it will prevent their use in places such as factory floors where AR will be very impactful, because a suddenly blinded worker next to heavy machinery is usually a terrible idea.
But I also appreciate how they solved the transparency issue of AR by instead of overlaying stuff on the real world, they just replaced it with an Apple-made one, which is insanely clever and totally Apple.
I also want to know what they can do with additional sensors - extending vision into IR and UV and beyond will certainly prove interesting, as well as providing better perception at low light.
The key thing is that it’s not a head set for an existing computer, but a computer with a new shape, with an OS designed for that. We shouldn’t discount the initial problems - this thing is impressive. It’s a quantitative improvement so large it will become a qualitative one.
And I’m not sure what you mean by “compared to VR” - do you watch some kind of 3D movies in VR? These are very different experiences to me. In the theater it’s just an aid for immersion, a full 3D environment wouldn’t allow for traditional cinema direction to happen.
I spent a few years in the early days of the VR rebirth doing live action VR. The issue you mentioned was one of the biggest problems for me. When the camera is locked down in a stationary position, the issue is not as noticeable. You can look where ever when ever you want. Just because the other people are talking to you does not mean you have to look at them.
However, no modern director wants a locked off shot, and wants the camera to move to add _________ to the shot. You've now removed that choice in the environment for the user. If the user is looking in a different direction when the camera starts moving, it is very unsettling. The user is no longer in control of this environment that is meant to give the user control.
You aren't going to be able to pause and get up and see what is inside the briefcase in a 3D Pulp Fiction remake. You won't be able to watch Jaws entirely from the point of view of the shark. What is being shown in the shot is the story being told.
People can and do make fully interactive experiences where you could do that. I would argue once you let the audience all walk around on the stage and asking if they can go back to the previous scene in the middle of the performance, you're no longer putting on a play. You're doing something entirely new.
Sorry, until the hardware is available I am not willing to sit down and read/listen about how terrible it is.
I have been wrong about Apple’s ability to execute before.
Mounting an external battery pack to the back strap of the Quest headsets has always been a common mod and it feels bizarre for Apple to ship theirs with an external battery on an umbilical cord that you have to manage as a user, as it's an extra hassle that ruins immersion and portability, especially that since I live in a warm climate, my pairs of underwear and shorts I wear in my house have no pockets as I don't need to carry my wallet, phone or keys in my house. What then? Do I hold the battery pack with my teeth or between my ass cheeks while I slice cubs in beat saber?
It feels like they prioritized some superficial design goals for the sake of looks, and didn't really think this through enough in terms of ergonomics by actually dogfooding this in users homes, just like their sleek good looking but uncomfortable and stupid wireless mouse that's the only one I know with the charging port on the bottom instead of at the front so you can't use it while charging like every other rechargeable mouse because of Apple's bizarre design choices.
I wish I were a fly on the wall in the boardrooms when such design decisions are green lit. Is it the product designers that have the final call? Is it the engineers? Is it a vote by committee or is it one person who has the decision power?
I don’t think the demo was pushing fitness games. I reckon Apple are comfortable leaving the $500 end of this opportunity for Meta and co.
I bet they included the power brick as a last resort and after thoroughly testing the weight of other mounted options. Insane to think otherwise.
Mostly sitting on a couch, or leisurely strolling in your huge, mostly empty, living room while discussing with your colleagues or family.
Instead of, say, wildly moving your arms around to slice a cube in beat saber, or shoot down a face hugger in Alyx.
The other big difference being the supposed ability to clearly see your environnement.
But then, this is all suppositions. And they have screwed up more than a few times. Wait and see, I guess.
So, 3500$ for a VR Zoom call machine?
Considering some people’s use cases, the same could be said of a MacBook Pro or an iMac.
I mean, have you seen their introduction video? [1] It’s basically watching movies on a huge screen and chatting with heads embedded in floating windows.
In the past, but even today in some parts of the world, it was a common practice to carry great weights on the head for a long time, with the condition of them being well balanced (because it was more comfortable than carrying them in an unbalanced way, in the arms or on the back).
I don't want to have a battery pack in my pocket and dangling cable while I'm in VR. Oculus could solve this on a $350 device. Surely paying 10x more could also have this groundbreaking feature.
I will stick with my Oculus Quest and aftermarket rear battery pack though.
Of course it can be solved with some ducktape, but for $3500 do you fell that's a great UX for your money? That's like buying a Mercedes S-Class and having to manually wind the windows.
Well yes, they obviously didn't think through apparently, because, and this might come as a shock to Apple designers in Cupertino, but not all of us are Californians sitting in our air conditioned homes wearing long jeans pants and Patagonia jackets indoor in high summer because our home AC is set to a frosty 20C (68F).
In my part of Europe, July and August are unbearable and there's no residential AC in apartments here, so sitting naked at home or in underwear is the only way to make it through these torrid summer months.
Also, I wasn't being snaky at all, Apple is not the only Bay Are corporation to designing consumer products around the climate and lifestyle of the people working for them, omitting the other global demographics that face different climate challenges and lifestyles of those in California.
How much are you willing to bet, that the second gen will feature a better battery support as the first gen will get criticized for this?
This is a very odd take. Southeast Asians and Louisianans work in conditions that are technically considered "uninhabitable" by OSHA. The response isn't working naked. It is not a use case that any company would consider primary.
Also, you know, you can always put on a belt without clothing.
Maybe I'm giving Apple a bit too much credit here, but it feels like they've given this some thought and decided to not go with that based on some sort of rationale, not oversight.
Was it the same rationale they used to ship the broken butterfly keyboard for 4 years straight, or for the uncomfortable bottom charging mouse that even hardcore Apple fans don't use?
It's always funny to see people making excuses for the poor decisions of their favorite corporation as if Apple can do no wrong and it isn't capable of making stupid decisions just like any other big corporation subject to the same internal turf wars and power struggles.
Both the Vive and oculus had lots of third-party support - I don't see that this is gonna be any different.
PC gamers have had to deal with umbilicals for a long time, and it isn’t that bad so long as you don’t move much. Of course, Apple would never say “it’s fine because PC gamers do it”. But it is probably fine.
Agree 110%.
Its not on sale yet.
Only a tiny handful of Youtube influencers have been allowed to have a sneak-peak, and even they were not allowed to film it or go into too much detail. So clearly they were being shown late-stage prototype models and not the finished product.
Its therefore clear anyone sensible should wait and see.
It's carefully argued, based on published specs with clear caveats around the more speculative parts.
It's worthy of engaging with on it's own terms.
And I have no idea why you are talking about dropbox and flying.
Apple's new device will not fail because there was some comment in 2007 on HN related to dropbox that pointed out alternatives (but even that commenter said it's good idea for windows).
I also caved and bought a pair.
The smart watch used to be pretty obscure and known as being a tech geek device, but now I see most people wearing an Apple Watch as opposed to not even wearing a watch it all just a decade ago.
as for apple vision ar is the future no doubt, that seems obvious like wireless earbuds did. Vision pro tho imo is a devkit, but the fans will buy in anyway id guess. wait till they release a more consumer focused model and then itll be more obvious AR is working out.
- lag is lower than other devices, perhaps low in absolute terms, but is it low enough? Perhaps not for all use-cases
- there are difficulties aligning virtual images (this part is very speculative as to how it relates to Apple’s headset design)
- difficulties tracking saccades - not sure under what use-cases this would show up as a problem, this hasn’t been mentioned in reviewers’ first impressions
- safety problems with obstruction of peripheral vision when moving around
Will it be useful though? It doesn’t do games or “presence”, or indeed seem very focussed on any uniquely AR/VR experiences right now. It’s not as good a monitor as a proper monitor (this analysis of VAC and resolution is pretty convincing). But it is portable. Maybe that in itself is enough?
Karl argues that the promised 12ms latency for video passthrough needs to be added on top of camera acquisition and display timing. This could lead to potential delay of up 3x 12ms in the worst case and 12ms in the best case based on 90Hz displays and 90Hz cameras.
Time will tell.
Nobody knows what is possible with AR. There are lots of ideas, none tested in a large enough group of consumers. Enter Apple, which can attract both a sizeable group of developers and a relevant group of consumers. What we'll see now is the divergent part of software development for a new platform. Everyone trying new stuff, finding out what sticks, what gets market fit.
It'll be fun. Then, a new device will refine the concept.
> Low Processing Lag Time and High Frame Rate are Necessary but not Sufficient to Solve Visual Issues
I think I read somewhere that Apple are using the eye tracking to concentrate processing power and refresh rate on the area you are looking. Eventually a higher refresh rate in a smaller section of the display.
> Poor location of main cameras relative to the user’s eye due to the Eyesight Display
> The proper location of the cameras would be coaxial with the user’s two eyes.
The eyes have two degrees of movement, the camera can't be "coaxial" with them at all times.
The proximity of the cameras to the axial direction of your eyes is going to be more important for short distance than long as the effect of misalignment is going to be stronger. We tend to be looking somewhat down when working on things near to us, and in the case of the Vision Pro that's where the cameras are. You are almost exactly looking down through the axis of the cameras when working on something on a desk or table in front of you. This is better than if they were horizontally in front of your eyes.
I don't doubt that Apple have considered all these issues in the design, and any trade offs they may have made. Once we get our hands on hardware we will be better informed how well they have addressed them.
That technique is Foveated Rendering, and my understanding is that it doesn’t change the frame rate. I think if it did you would get obvious visual tearing at the boundaries of the different areas. I believe it normally involves just rendering areas at a lower resolution and then applying a basic blur to it.
Additionally, I’m not certain if that will be applied to the pass through video, the most important bit for these visual issues. Adding foveated rendering to that pipeline could even do more harm than good in terms of adding latency, and video processing is typically much faster than the rendering of new graphics from scratch, particularly given the M1’s dedicated video accelerators.
They could gradually blend the area with higher refresh rate to the peripheral images with lower refresh rate. You would probably not notice it because the resolution in our peripheral vision is terrible. Even with sharp tear lines it might not be noticeable if the frame rate is fast enough.
I don't think this is what they're doing, but it could be viable.
I think you're right about the other issue with foveated rendering of pass-through video. You really don't want to do much processing.
But imagine this: if you can control which line the camera starts on with its rolling shutter, you could start the camera and the display update a few lines above the point where your eye is focusing. You can blend the first few lines with the ones from the previous frame to make the tear line less obvious, which would require almost no processing. They may be processing a few lines at the time before sending it to the display for color correction, simple transformations, etc. Could be viable.
I'm just not sure that's possible for update frequency. For lower/higher resolution, sure, but even with our terrible peripheral vision I think we'd notice that the center of our vision is smooth while our peripheral vision is "sticking", particularly during rapid movement.
It's called foveated rendering. Everyone following the space should listen to carmack's keynotes. Whe gove over a lot of stuff there and it's really honest and, IMO, technically meaty.
It could be social conformance. Nobody wants to be the first person to do something conspicuously different in a constrained setting like an airplane.
I remember when I got my first mobile phone sometime in the 1990s. It felt embarrassing to answer it in the middle of the street and carry a private conversation right there. Today there are essentially no boundaries to where you can use a phone.
Apple can easily afford to spend billions on ad campaigns and influencer marketing to make headset use behavior look acceptable and get over the social conformance threshold. No guarantees that they can pull it off, but they have the resources to try. The external EyeSight display on the unit seems to be a big part of this.
Her review is almost entirely orthogonal to Karl's. She focuses on the efficacy of the experience. Apparently Apple is able to sidestep a lot of the issues that Karl is mentioning. For instance all the vergence issues seem to be non-existent in her usage. Likely because Apple isn't focusing on interacting with any real world objects.
It’s prob either that they want to package the tech as their own and give it fancy labels (like spatial computing). Or they are concerned about being compared to competitors. Or perhaps they want to do the “it just works” thing where you are not supposed to care about the specs, like system memory on iPhones. Maybe all three.
But if they want to avoid being compared to competitors, that concerns me. As they use that strategy to deliver inferior products.
A good look at the topic, it's nothing new for them. They specifically love to both avoid marketing terms for some things and to give their own specific branding terms to others (think "Retina" instead of High Res, "ProMotion" instead of 120Hz, etc).
Based on the initial reviews, I think it’s more the opposite: they don’t want you assuming that their product has the same limitations which have characterized those acronyms so far. From what I’ve read they have gotten over a couple of key thresholds for resolution, response time, etc. which held back lower-spec products and don’t want people to think prior experience with e.g. Oculus devices is representative.
I think they might want to avoid these spec-to-spec comparisons because other aspects of implementation can give a better UX (like software in iOS compared to Android). Though, until we see the hardware, it will be difficult to say whether they can pull more out of current industry-leading specs than others.
I don’t think it’s unprecedented on the raw hardware capacities but the combination is: the Pimax is only a VR headset, so it’s missing half of the hardware, and it has half the PPD; the Quest 2 can go higher on refresh rate but has much lower resolution, no eye tracking, etc. while the Quest Pro gets closer on some of those features but at under a third the PPD.
I mention that because one thing which has stalled AR/VR adoption is the high threshold your brain sets for accepting a scene as real, especially for AR. We won’t know until it ships but I did find it interesting how many reviewers mentioned this being the first time things came together. It might be that much of the conventional wisdom around AR is too pessimistic and it’s simply that you need to reach an aggressive combination of resolution, color depth, responsiveness, tracking precision, weight, etc. similar to how it took decades for digital cameras to hit the quality/weight/cost thresholds where the benefits became compelling. I wouldn’t be at all surprised if it turned out that the Vision Pro is barely over that level for Apple at a high price point and out of reach for Facebook for a few more years since they don’t have a high-end CPU team in house and have to wait for the commodity market to advance.
But that doesn't solve "accidentally disconnecting the battery"
IMO that’s exactly how the article approaches it: Here are some known issues where we can already see that Apple is taking a design approach that’s known to be problematic. I didn’t have much problem with the tone. You find harsher words and less qualified arguments in the typical “I don’t like this or that JS framework” programmer rant that makes it on HN.
I'm glad that this review explained more of the downfalls of this design choice before I plunked down the credit card.
Oh you are hilarious.
First let's cover "unnecessary heat, battery usage". The headset is based around Apple Silicon. And as Apple has perfectly demonstrated with their Apple Silicon MacBook range they are very much the market-leaders in eliminating unnecessary heat and maximising battery usage. I suspect we will find the same with the headset, I doubt there will be a problem.
> EM radiation
Oh right. Because VR headsets (from ANY manufacturer) are so amazingly useful without Bluetooth or WiFi enabled ! if you're worried about EM radiation, then don't buy one ... oh and whilst your at it, throw away your cell phone, computers, laptops and build a faraday cage around your house.
I'm sorry but I can't show sympathy for EM radiation concerns of VR headsets when you are clearly using 101 other devices in your every day life that emit EM.
> social awkwardness
That applies to anyone wearing a VR headset from ANY manufacturer on their head.
Whether the front of the unit is Apple's eyes or Meta's plastic, it's the same thing .... a weird contraption siting yon your head.
It would be a hoot to see people write modifiers for the displayed eyes, to show things like cat pupils.
As for the rest, we’ll have to see. There’s a long history of really difficult problems with 3D/VR/AR, but Apple has more money than some countries, so, if anyone can solve them, they can.
I remember when the iPhone came out. I worked for a camera manufacturer, at the time. One of my employees brought one of them, and I looked it over. One of the things that immediately jumped out at me, was the built-in rear camera. It was pretty limited, with crappy resolution, poor light-gathering, weird depth-of-field, shaky image quality, and noticeable latency, but you could preview the image, live, on a fairly big screen.
People may also remember, that, when the iPhone first shipped, there was no SDK, and the App $tore did not yet exist. No one could write any apps for the phone. The JS Web apps thing sucked (even though many folks now write iPhone apps in JavaScript). Even Steve Jobs couldn’t make developing for iPhone attractive.
Nonetheless, using the device was incredibly convenient. I knew that if they could improve these issues, it would be a winner.
I mentioned it to our marketing folks, and was laughed out of the room.
A few years later, our entire consumer camera division, which had made us billions of dollars, was in the toilet. Even our pro camera division took some serious hits. A couple of major newspapers fired all their pro shooters, and told their reporters to use iPhones.
Sometimes tech doesn't go anywhere but a whole lot of promises..
This is why I tell everyone in my company not to complain about something unless they have a better solution.
Does this guy offer any solutions or present better products? No. That's because with current technology, this is the best we can do.
What a jerk.
Not saying this is the case with the Apple Vision Pro.
I’ve had enough of this pompous and arrogant company. This is going to be good.