pandaforce

Bellard hasn't been involved in FFmpeg for *over 20 years* at this point, and more like 23. His code was not great and reeked of sphagetti due to FFmpeg back then lacking any framework for code sharing between components and codecs. These days none of his code survives. Everything that became of FFmpeg is because of other developers. Yet he's treated as the one-and-only BDFL of FFmpeg, with any other developers building upon his wise framework since time immemorial. These days all he does is hold the copyright, which lets him, *and only him*, elect which project/leader may call itself FFmpeg. He's an unelected dictator, who already used his powers once to ostracize libav developers in favor of another dictator.

Yes. I get 300fps decoding 8k ProRes on a 4090 and barely 50fps on a Zen 3 with all 16 cores running. The CPU doesn't touch anything, actually. We map the packet memory and let the GPU read it out directly via DMA. The data may be in a network device, or an SSD, and the GPU will still read it out directly. It's neat.

Most vision models are trained on images or conventional video codecs. There's a good reason why H200's have 7 JPEG + 7 nvdec ASICS.

That's basically what VC-2 is, as well as JPEG2000-HT. Wavelets are nice in that they have no need for deblocking since each slice complements one another, and they're simple to calculate. But this localization means they're horrible for compression since they have bad frequency decomposition properties. JPEG2000 showed that.

Proxy files have issues when doing coloring, greenscreens, effects shots. The bit depth, chroma resolution, primaries/transfer/colorspace gets changed. Basically only really usable when editing. With this, you don't need proxy files at all.

FFv1's range coder has higher complexity than CABAC. The issue is serialization. Mainstream codecs require that the a block depends on previously decoded blocks. Tiles exist, but they're so much larger, and so rarely used, that they may as well not exist.

> A crash mid video write out can corrupt a lengthy render. With image sequences you only lose the current frame.

You wouldn't contain FFv1 in MP4, the only format incompetent enough for such corruption.

Apple has an interest against people using codecs that they get no fees from. And Apple don't have a lossless codec. So they don't offer lossless compressed video acceleration.

The idea is that when working as a part of a team, and you get handed a CG render, you can avoid sending a huge .tar or .zip file full of TIFF which you then decompress, or ProRes which loses quality, particularly when in a linear colorspace like ACEScg.

6k ProRes streams that consumer cameras record in are still too heavy for modern CPUs to decode in realtime. Not to mention 12k ProRes that professional cameras output.

The main target for this are NLEs like Blender. Performance is a large part of the issue. Most users still just create TIFF files per frame before importing them into a "real editor" like Resolve. Apple may have ASICs for ProRes decoding, and Resolve may be the standard editor that everyone uses.

But this goes beyond what even Apple has, by making it possible to work directly with compressed lossless video on consumer GPUs. You can get hundreds of FPS encoding or decoding 4k 16-bit FFv1 on a 4080, while only reading a few gigabits of video per second, rather than tens and even hundreds of gigabits that SSDs can't keep up. No need to have image degradation when passing intermediate copies between CG programs and editing either.

Yes, but no. No, in that these days, GPUs are entirely scalar from the point of view of invocations. Using vectors in shaders is pointless - it will be as fast as scalar variables (double instruction dispatch on AMD GPUs is an exception).

But yes from the point of view that a collection of invocations all progressing in lockstep get arithmetic done by vector units. GPUs have just gotten really good at hiding what happens with branching paths between invocations.

Yeah, Vulkan is shedding most of the abstractions off. Buffers are no longer needed - just device addresses. Shaders don't need to be baked into a pipeline - you can use shader objects. Even images rarely provide any speedup advantages over buffers, since texel cache is no longer separate from memory cache.

GPUs these days have massive cache often hundreds of megabytes large, on top of an already absurd amount of registers. A random read will often load a full cacheline into a register and keep it there, reusing it as needed between invocations.

The article explicitly mentions that mainstream codecs like H264 are not the target. This is for very high bitrate high resolution professional codecs.

These are all gripes you might have with Vulkan Video. Unlike with Vulkan Video, in Compute, bounds checking is the norm. Overreading a regular buffer will not result in a GPU hang or crash. If you use pointers, it will, but if you use pointers, its up to you to check if overreads can happen.

The bitstream reader in FFmpeg for Vulkan Compute codecs is copied from the C code, along with bounds checking. The code which validates whether a block is corrupt or decodable is also taken from the C version. To date, I've never got a GPU hang while using the Compute codecs.

Khronos published a post on the Vulkan compute codecs in FFmpeg: https://www.khronos.org/blog/video-encoding-and-decoding-wit...

Cheaters are always 2 steps ahead, all that developers can do is to trim the less willing ones.