Hacker News Reader

Comment by epeus | Hacker News Reader

Yes they are, because existing video codecs are based on 1980s signal processing ideas, and increasingly video starts out as being 3D models that are naturally better suited to a trimesh and textures approach instead. I wrote about it here: http://epeus.blogspot.com/2013/05/finally-some-progress-in-v...

View on HN

Gleaning useful 3d models from 2d images is rather error prone. Hence why all the cool kids are using wavelets, as they are inherently more reliable on 2d images.

Movie codecs are hard. However jpeg2000 and similar wavelet codecs (like Red's(those bashful camera people) REDcode, or dirac) can yield amazing compression ratios. The main problem is processing power.

Take Red cameras for example. When they first came out, it was a massive pain. Unless you spunked $5k on a super unreliable pci card you'd only get a maximum of 6 fps out of a 8-core mac (and it could only be a mac as there was no linux SDK) With the advent of 3 generations of CPU and better programming skill from people like avid, aja and the foundry we can get real time playback on a laptop.

Using wavelets for encoding the textures makes sense. The challenge with JPEG200 is that you need the decoder in the read loop to know when to stop getting more detail. The degenerate case would be 2 triangles and a texture, but if you have existing world models, decimating the meshes and de-rezzing the textures appropriates would work. Read m post, it's still a Graduate Student algorithm, but cut scenes on consoles are an existence proof.

I understand that wavelet is amazing, but am a little sceptical about it being used outside of a few specialist areas. The general trend seems to be towards using increased storage and power to avoid compression (raw), rather than making files smaller.

The thing that replaced 35mm with a digital format is DCP. Basically its Jpeg2000 in disguise.

The problem with uncompressed formats at high resolution (like cinema 4k, and its smaller "UHD" cousin.) is that they eats bandwidth. Full res 4k images with basic RLE is 50megs. Thats 1.2 gigabyte a second. You're lucky to get that out of 10gig Ethernet.

Uncompressed anything is pretty much out of the reach of most consumers. iTunes, netflix, over the air, cable is all either MPEG2 or h.264

But surely nothing has really changed over the last 15 years. Processing power, storage, and bandwidth all get larger, but so do the images. Wavelets must be a great technology if you want to fit a HD video on a CD and play it with an expensive machine. I am not arguing against compression. But you have to admit that Jpeg2000 has been the next big thing for a long time. Surely it is cheaper to just use h264 and have more storage?

Depends on the application, In broadcasting for example bandwidth is painfully expensive, hence the shift from MPEG2 to h.264 for consumers (although in the US its much more complicated as everyone seems to recompress everything badly on TV. Its like watch a pirated copy.)

It just takes a long time to get from approval to acceptance. The other problem with JPEG2000 is that its not really a motion codec, so it doesn't degrade well over lossy links without special sauce. All of which is non standard.

I think this thread proves a point about compression. That it is about balancing different requirements rather than absolute performance in any one measure.

What are you talking about? Film is 2D. How do you imagine it to suddenly be "encoded in 3D"?

Object-based encoding has long been discussed within the MPEG community. MPEG-4 included VRML as a sort of hack for scene based compositing and Structured Audio Orchestration Language et al, as a parallel.

I saw early whitepapers where they had planned even to do things like recognize people's faces, and deduce geometry, thereby separately compressing a person in the foreground vs background (use case: conference calls, news anchors, etc)

As for whether it is possible to deduce 3D structure 2D, there's been some progress in that regard, check this out: http://www.youtube.com/watch?v=CZiSK7OMANw

Using 3d space to represent movement of the textures. Currently a frame is split up into blocks, and they are moved about in 2d (X&Y). Using a 3d model of the movement would theoretically gain you some savings, as it would require less refreshing of the moving blocks.

Quite. You can define a textures+trimeshes+motion format and have the decoder be well defined, then have encoders compete to do better, as we have seen with the 2d codecs. Clearly starting form something in GL, like a game walkthrough (How much of youtube is that?) is easier than starting with Bladerunner. A Pixar movie would be somewhere inbetween.

The world is 3D. I bet the last few films you saw had extensive 3D modelled scenes in them. I bet all the video games you play have 3D cutscenes.

This is true, but the are pre-rendered for a reason. Anything with water you're looking at 24hour+ sim time, plus render, and 1tb of data plus. SubSurface Scattering (translucency) takes a huge amount of CPU time.

The world is 3D. Video isn't.

Deleted comment

Would companies that produce computer animated movies be willing to risk exposing their 3d/texture assets like that? If even low-quality "pre-rendered"^ versions of those fell into the hands of the community then those companies would basically have to deal with a bunch of people using their stuff (probably mostly fair-use covered parody stuff on youtube, but also high quality Chinese knockoff movies and the like.)

^ You would have to pre-render that stuff in some way first, wouldn't you? The actual source assets for those movies are absolutely massive and do not render in real time.

Reply on news.ycombinator.com