Hacker News Reader

Comment by MBCook | Hacker News Reader

You want a killer app for VR/AR goggle style things? You’re right this would be amazing.

Apple demoed some kind of volumetric video to the press with the Vision Pro. There was a short clip of a concert and an NBA game (Nuggets?) among other things. I heard a number of people said it was like being there.

This is a step past that. Apple’s was recorded from some kind of special camera rig (I assume), but I seriously doubt it was full volumetric video from a large number of angles. It sounded more like volumetric video if you were stuck in a (very good) seat in the venue.

I’d be curious to know just how much horsepower it takes to play these back.

View on HN

I thought they recorded their special videos using the Vision Pro itself, which has enough sensors to build depth maps of the scene and provide novel views within a small range from the original position.

But I am half speculating and I don't really remember. That's just the impression I remember having.

That’s a feature of the headset, but I think some of the demo videos were recorded in some other manor. I seem to remember hearing a discussion on a podcast (The Talk Show? Dithering?) where it was mentioned you could see a camera rig somehow.

Perhaps that was something you could see on TV when they accidentally showed the rig that was at the same event?

Of course it’s possible they were only using a camera rig so that no one would see the device before they were ready to invade it. Which would be very Apple like.

So I’m speculating a bit. But even if they were going to professionally record events I would think they would do better than just have someone sit there with a headset on.

I saw the image you’re referencing yesterday. The unit appeared to be a plane, briefcase sized. It had two fisheye lenses about 10cm diameter each and separated by what I expect is an average human IPD.

So I think that unit was simply capturing 180 degree stereo video. Not enough to compute volume without most of it being some ML inference.

It’s not full volumetric video. Probably can simulate a few degrees off axis but nothing like what this article shows.

From the paper, it seems a 3060 is enough for 60FPS for the DNA-Rendering dataset. On the full-screen datasets it manages 25 fps. A 4090 might be needed to stay above 60 fps.

Still pretty heavy I’d say but it certainly came a long way and shows us real volumetric video is doable.

That's real-time rendering though, right? Is there anything preventing it from being pre-rendered in non-real-time first? Or does it have to be rendered in real-time?

I'm not familiar with any of this at all, so I'm genuinely curious.

If the context was to use in a VR/AR headset it has to be real-time. And I guess that use-case, and the related that you interactively want to walk around a scene are two of the main use-cases

I think the best way to consider it.. is as a 3-d cloud of individually addressable pixels. The size of the cloud is dimensions of real-time rendering.

The interesting bit is that it's feasible to render in realtime. We have plenty of ways to render a 3D scene if render time is not an issue.

Reply on news.ycombinator.com