Facebook Surround 360: An Open 3D-360 video capture system | Hacker News Reader

Facebook Surround 360: An Open 3D-360 video capture system

code.facebook.com

144 pointsrunesoerensen53 comments

360 video is cool but feels a little gimmicky for personal use.

180 degree 3D video, on the other hand, is AWESOME - that's the true-life look/feel that I think is going to be game changing and sell another 100m units for GoPro or whomever comes up with it.

So far the players I know are:

http://vuze.camera/ https://webeyevr.com/ http://lucidcam.com/

Or some combo of kit that you can cobble together: http://shop.gopro.com/accessories/3d-hero-system/AHD3D-001.h....

Or run a DIY game: http://www.stereoscopynews.com/hotnews/3d-technology/softwar....

Anybody have any experience to share on a good consumer balanced solution?

I think they called out the target audience being pro applications. This is a great opportunity to give someone like the NHL/NCAA/etc ability to stream 360 to a facebook base.

I guess that announcement makes sense for professional content producers, but this is mostly a clickbait product. One that we won't probably see in real life, although it's clear that VR has pushed interest for 360 videos.

Good as an option, probably, but most of the time flat video is ideal. We'll see if that changes with these products or the ones made for big audiences (Samsung Gear 360, and LG Cam 360). I wouldn't bet my life on it. Adventure cameras have already reached their limits, it seems, for example, and I see a similar scenario on this case.

I am not so sure. I once played around with projecting an imax 3d space movie on the inside of a virtual sphere. It was a pretty awesome accident, because (so long as I didn't turn around and look at the seam) I had the illusion of asteroids and stuff actually floating in my space. I think there is a lot of potential for 360-3D video.

That said, I am not sure that content producers should focus too much on having things happening all around you. It's not great to constantly have to be looking all around, but it is useful to create the illusion of presence.

Indeed. Cheers!

What's notable about this announcement is that it claims to be 360 degree _3D_ video. The article talks about this. That means it's like a 180 degree 3D video in every direction.

That said, I'm a bit skeptical about how well their method works. If you close one eye and rotate your head, you'll notice parallax effects, and I'm not confident that what Facebook has described will be able to reproduce that. But I'd love to be proved wrong.

They generate depth using optical flow and then generate separate images for each eye.

If you model the camera in 3D space within a sphere that the video is "projected" upon, would you still notice those parallax effects when closing one eye and looking around?

It's pretty nifty on a drone. Instead of just shooting some small field of view, fly a path over a cool area and let the viewer decide where to look.

I don't remember family holiday videos being that cool when i was younger!

Two a6300's next to each other with Samyang 8mm fisheyes does the job nicely for 3D180

The best bet as of today is an array of GoPros, same as for most 360 solutions. You can get about 60 degrees out of each lens after accounting for stitching overlap, so six of them are sufficient for a 180 degree stereoscopic picture. I've gotten some good results with this, though of course there's a lot of black space, which a 12+ camera rig would alleviate.

I couldn't disagree with this more. For me, 3D is a gimmick. Always has been. Making it a wide field of view doesn't change that.

360-degree video, on the other hand, adds a crazy new element to how you experience things and how you shoot. It's easy to upload to Facebook and YouTube and it can be played back on any device. The coolest part of a 360 video is when you are standing in the same spot that it was filmed. It's like being able to travel back in time to when it was shot.

The coolest part about 360 video, in my opinion, is that it doesn't matter so much where the camera is. With enough resolution you can zoom crop into a standard frame. With just 2 cameras you can create all new kinds of linear content just by panning and cropping around the sphere.

It's mostly about the realities of the stitching though. 180 3D is usually done with two cameras facing forward with a good lens to capture the maximum from each side.

360 3d is a huge lie in reality, although when it's well done (Felix&Paul studios come to mind), it can be quite pleasing.

I'll be doing both 180 3d and 360 for a friends marriage. I'm using the 180 3d to have the guests come and leave the newlyweds a personal message. Everyone will come wish them good luck and lots of love etc.

Then the 360 shots are fixed and capture the feeling of the moment, like reliving it. The 3d effect on 180 feels better and your focus is guided too.

Anyways just my 2 cents.

Edit : Just noticed your post was about 3d vs non 3d for the most part.

I think 360 video is great for personal use. Earlier this year I bought an $350 Ricoh Theta S and shot some 360 video. Here is an example of my friend [1] colliding into me while skiing (might want to turn the sound down though, sorry). With just a 180 camera you wouldn't be able to see what happened and why.

3D cameras are great, but 360 takes you a long way toward immersion when used in a headset. Stereoscopy works well only when the objects you care about are between 6 and 20 feet from the camera. Outside of that stereoscopy doesn't give anymore depth cues simply because of how close together our eyes are. Here is a photo wigglegram [2] I took with a $20 Nimslo 3D; it's widest lenses are as far apart as your eyes. Note how flat the background starts to look after the second row of people? That is because they are too far.

As a weird camera and VR enthusiast, I have found it non intuitive to predict what will work or not. I hope this facebook rig and software that uses "optical flow to compute left-right eye stereo disparity" will be immersive and practical. 360 in 3D facebook and google promise sounds fantastic. In a few years, I hope we will have something for the non-professional content producer.

[1] https://www.facebook.com/fooladder/videos/vb.8800136/1010602... [2] http://fooladder.com/post/115435676962/at-the-concert

How does this compare to Jump for Cardboard & GoPro Odyssey from 8-10 months ago?

https://www.google.com/get/cardboard/jump/

https://gopro.com/odyssey

https://news.ycombinator.com/item?id=9620224

As far as I understand this Odyssey is GoPro building the actual hardware and Google Jump is the software technology that powers it. In terms of goals, it's pretty much the same as Facebook's new camera.

Other players in this space are Lytro [1] (they have pivoted away from consumer cameras to professional VR capture hardware) and Jaunt [2].

[1] https://www.lytro.com/immerge [2] https://www.jauntvr.com/technology/

Someone correct if I'm wrong, but I'm pretty sure they're using the Grasshopper3 camera, which retails for about $3k, and there are 14 of them in the pictured array. That's $42,000 just for the cameras. I know professional video shooting equipment is expensive, but that's more than double the price of a top of the line RED Epic-X camera that a professional 2d film crew might shoot on.

So far as I can tell from the scant technical details, this isn't a true 3D 360deg video solution. It seems like this camera will provide stereo 360deg video for horizontally aligned left-right eye positions, which is not the same as 3D 360deg video that supports any head and eye poses (that requires full light field capture or reconstruction). This means any user viewing a video that rolls their head left or right several degrees will get a headache pretty fast. I'm sick of people claiming '3D' 360 video is the same as full 6DOF tracked VR - it is not. </high horse>

It's hard for me to get excited about stereo 360 when we have light field capture coming soon.

This feels like the era of 640x480 digital cameras... It's easy to see the potential, and the current gen is still valuable for niche uses, but it's clear the next generation is what will really deliver on the promise of immersive video.

Light field capture coming soon? I'd like to learn more.

https://www.lytro.com/immerge

Light fields are definitely the future but big challenges with huge data rates. I don't know how soon "soon" will be for consumer devices. OTOY is doing cool work with light field streaming

Check out Lytro Cinema:

http://techcrunch.com/2016/04/11/lytro-cinema-is-giving-film...

https://www.lytro.com/cinema

I didn't find the link for all the specification about how to build one at home. Is it really open source ?

It is not open source. In the article, it says,

> later this summer [Facebook] will put the hardware designs and video stitching algorithms on Github

I do think it will be open source eventually, but it doesn't seem to be right now. We'll see.

Hopefully it doesn't come with homegrown not-actually-free-software license that their other 3D video repos had. https://github.com/facebook/transform

360 video is cool but feels a little gimmicky for personal use.

180 degree 3D video, on the other hand, is AWESOME - that's the true-life look/feel that I think is going to be game changing and sell another 100m units for GoPro or whomever comes up with it.

So far the players I know are:

http://vuze.camera/ https://webeyevr.com/ http://lucidcam.com/

Or some combo of kit that you can cobble together:

http://shop.gopro.com/accessories/3d-hero-system/AHD3D-001.h...

Or run a DIY game:

http://www.stereoscopynews.com/hotnews/3d-technology/softwar...

Anybody have any experience to share on a good consumer balanced solution?

Is there a VR headset out there that already allows you to experience 3D?

Yes, all of them. Oculus Rift, HTC Vive, Google Cardboard, etc. They show a different view to each eye.

Depends what you mean. 3D/360-degree? It's sort of doable using cameras and software to stitch and interpolate all the different angles but you're still viewing from a fixed perspective. That works in the current crop of VR headsets. Same with 3D/180-degree (or similar wide angle but not 360 3D stuff). I've watched several on my Rift dev unit.

Getting into "real" 3D, like being able to view from any angle is trickier but there are experimental and in-development methods of doing this. The easiest works for CG stuff (just look at your 3D content in your headset). The more realistic it gets, the closer to a "movie" in 3D. Combining video footage with CG environments is a nice little "cheat" that's already been played with as well.

But the real challenge and (as I see it) the ultimate goal in this space is being able to use something like multiple light field cameras or RGB+depth cameras (think Kinect) arrayed around a room or environment and capturing both live video and the 3-dimensional structure of the surroundings and any objects or people. The depth data from the various angles needs to be combined and "textured" with the video data in order to create something approaching a real 3D recording of something over time (so maybe they'll need to advertise as 4D movies).

Even cooler is when you can compress all of this data and stream it over networks to be decoded on the other end. The result will be like 3D telepresence where you're all but transported to another location.

Microsoft has been working along those lines and it makes sense since they have Kinect and Hololens. This is the sort of thing that will be the real "killer app" for VR/AR as far as I can guess: https://www.youtube.com/watch?v=7d59O6cfaM0

The challenge, as you said, is the data compression. Well, that and optics; it's hard to grab all the photons someone might see if they moved around....

I was already blown by seeing the Oculus conference in California in my GearVR (Next VR app). This is the type of technology which makes me dream about how good retirement will be one day.

I own the ricoh theta s which is a $350 consumer 360 camera and love it. I take it to friends weddings and people lose their minds at how fun the 360 pictures are.

I think that if Ricoh makes a followup camera with better resolution, it should be HUGE. The current Theta and Theta S have already enabled a lot of people to "hack" with it :)

https://codetricity.github.io/theta-s/

The Theta S's quality is almost but not quite good enough for what I want to do, so I eagerly await the followup.

whoever /

Even though this is purely to fuel VR content creation for Oculus, at least the invention is in the public domain (pending OS license and patent grant)

Agreed. Really looking forward to the stiching code being made available on GitHub. I wish that Google would at least release some kind of an API for their Cardboard Camera (e.g. allowing you to pass in a bunch of photos and get a stitched 360 out), but if Facebook makes good on this, it could be even better.

Why so many cameras?6 or 8 aren't enough?

I would imagine the stitching required a good bit of overlap of a video with its neighboring videos.

Also for the 3d every direction needs to be visible by at least 2 cameras.

You need multiple cameras seeing every image point so your CV algorithms can have a good depth reconstruction -- and so it doesn't break if one camera has a lens flare. >2 viewpoints also helps with reconstruction ambiguities, etc.
Google Jump uses ~16 cameras for the same reason.

based upon the photo-spheres i've made, the more pictures definitely help correct light-levels. Especially correcting areas looking at sun or glare.

I'd like to see something that takes a depth map with lidar and then actually lets you explore the photo.

Lidar isn't needed. Light rays themselves contain enough information to rebuild the scene.

Which cameras are they using. Is this pointgrey grasshopper3? Is there a cheaper way to do this well? :)

How does this compare with the theta https://theta360.com/en/
It's much cheaper and available now.

Theta isn't 3d, it's like looking at the world with one eye closed.

Maybe a better analogy is that it's like listening to mono audio. Your analogy would be like putting one earbud in.

Great news for Point Grey.

Works only if live feed is sent to FB servers. /s

Facebook is the last company I would want to buy this from.

Completely agreed. Their bottom line is to collect all your personal information.

I feel like there's a group of people who have a list of these sentiments that they just copy and paste into any thread mentioning Facebook, adding absolutely no original thought or insight.

Well, they don't actually sell it, so you're lucky.

Reply on news.ycombinator.com