Hacker News Reader

Comment by doctorpangloss | Hacker News Reader

What is the use case? Okay, ultra low latency streaming. That is good. But. If you are sending the frames via some protocol over the network, like WebRTC, it will be touching the CPU anyway. Software encoding of 4K h264 is real time on a single thread on 65w, decade old CPUs, with low latency. The CPU encoders are much better quality and more flexible. So it's very difficult to justify the level of complexity needed for hardware video encoding. Absolutely no need for it for TV streaming for example. But people keep being obsessed with it who have no need for it.

IMO vendors should stop reinventing hardware video encoding and instead assign the programmer time to making libwebrtc and libvpx better suit their particular use case.

View on HN

The article explains it. This is not for streaming over the web, but for editing professional grade video on consumer hardware.

davinci resolve is the only commercial NLE with any kind of vulkan support, and it is experimental

prores decodes faster than realtime single threaded on a decade old CPU too

it doesn't make sense. it's much different with say, a video game, where a texture will be loaded once into VRAM, and then yes, all the work will be done on the GPU. a video will have CPU IO every frame, you are still doing a ton of CPU work. i don't know why people are talking about power efficiency, in a pro editing context, your CPU will be very, very busy with these IO threads, including and especially in ffmpeg with hardware encoding/decoding nonetheless. it doesn't look anything like a video game workload which is what this stack is designed for.

That reduces power consumption. So should improve battery life of laptops and help environment a little.

How do you figure? Power consumption will be higher. Decompressing on GPU with shaders uses the highest power state and every engine (the copy, compute and graphics for your NLE). The CPU will be running at full speed for IO and all the other NLE tasks. Using the GPU might be more efficient, but it will not matter to decode faster than realtime, which is my whole point.

> Decompressing on GPU with shaders uses the highest power state

Huh?

6k ProRes streams that consumer cameras record in are still too heavy for modern CPUs to decode in realtime. Not to mention 12k ProRes that professional cameras output.

How do you figure? Have you tried? The CPU is required for IO. Deciding ProRes is pretty simple, that's why you can do it in a shader in the first place, and the CPU will already be touching every byte when you're using Vulkan.

Yes. I get 300fps decoding 8k ProRes on a 4090 and barely 50fps on a Zen 3 with all 16 cores running. The CPU doesn't touch anything, actually. We map the packet memory and let the GPU read it out directly via DMA. The data may be in a network device, or an SSD, and the GPU will still read it out directly. It's neat.

50fps sounds greater than 24fps, which is greater than realtime, no?

> We map the packet memory and let the GPU read it out directly via DMA

packets from where, exactly?

another POV is, all the complexity of what you are doing, all the salaries and whatever to do this prores thing you are doing, is competing with purchasing a $1,000 mac and plugging a cable in.

if your goal is, use a thing in the Apple ecosystem, the solution is, the Apple ecosystem. it isn't, create a shader for vulkan.

i didn't say that streaming prores 8k doesn't make sense. even if it were 60fps. i am saying that it doesn't make sense to do product development in figuring out how to decode this stuff in more ways.

The article explicitly mentions that mainstream codecs like H264 are not the target. This is for very high bitrate high resolution professional codecs.

I'm not entirely sure that this is true.

I haven't actually looked into this but it might not be the realm of possibility. But you are generating a frame on GPU, if you can also encode it there, either with nvenc or vulkan doesn't matter. Then DMA the to the nic while just using the CPU to process the packet headers, assuming that cannot also be handled in the GPU/nic

You can also often DMA video coming in through peripherals to get it straight into the GPU, skipping the CPU.

> If you are sending the frames via some protocol over the network, like WebRTC, it will be touching the CPU anyway. Software encoding of 4K h264 is real time on a single thread on 65w, decade old CPUs, with low latency.

This is valid for a single stream, but the equation changes when you're trying to squeeze the highest # of simultaneous streams into the least amount of CapEx possible. Sure, you still have to transfer it to the CPU cache just before you send it over WebRTC/HTTP/whatever, but you unlock a lot of capacity by using all the rest of the silicon as much as you can. Being able to use a budget/midrange GPU instead of a high-end ultra-high-core-count CPU could make a big difference to a business with the right use-case.

That said, TFA doesn't seem to be targeting that kind of high stream density use-case either. I don't think e.g. Frigate NVR users are going to switch to any of the mentioned technologies in this blog post.

> This is valid for a single stream, but the equation changes when you're trying to squeeze the highest # of simultaneous streams into the least amount of CapEx possible... Being able to use a budget/midrange GPU instead of a high-end ultra-high-core-count CPU could make a big difference to a business with the right use-case.

okay well, the $5, 9 year old i5-9500T i benchmarked has 6 cores, and 1080p H264 decode is 2x realtime per thread, it would support 12 realtime streams. the scenario you are talking about, with frigate NVR users, well that thing supports a lot of ways to display many thumbnails simultaneously very cheaply - a human being can't even see more than ~3K resolution on a typical display at typical viewing distance - and it is pretty acceptable to drop frames, because the user isn't going to be able to comprehend the simultaneous, realtime video of more than 6 camera feeds.

i think you are proving my point that, this is all a bunch of overengineering. the ancient intel chip has QSV which will work just fine. nobody needs GPUs to be doing this work.

It will be more energy efficient. And the CPU is free to jit half a gig of javascript in the mean time.

It's hugely more efficient, if you're on a battery powered device it could mean hours more of play time. It's pretty insane just how much better it is (I go through a bit of extra effort to make sure it's working for me, hw decoding isn't includes in some distros).

It’s a leftover mindset from the mid-2000s when GPGPU became possible, and additional performance was “unlocked” from an otherwise under-utilized silicon.

Reply on news.ycombinator.com