Comment by PaulDavisThe1st | Hacker News Reader

Comment by PaulDavisThe1st | Hacker News Reader

Parent Full thread

PaulDavisThe1st

The last time I checked, the problem with GPUs and realtime audio is that although the GPU is extremely powerful and the bandwidth is fairly large, the latency for a roundtrip from/to the CPU is too large for what we typically consider "low latency audio".

Put differently, the GPU is plenty powerful to do the processing you're thinking of, but it can't get the results back to the CPU in the required time.

For video, there is no roundtrip: we send a "program" (and maybe some data) from the CPU to the GPU; the GPU delivers it into the video framebuffer.

For audio, the GPU has no access to the audio buffer used by the audio interface, so the data has to come back from the GPU before it can be delivered to the audio interface.

I would be happy to hear that this has changed.

None of this prevents incredibly powerful offline processing of audio on a GPU, or using the GPU in scenarios where low latency doesn't matter. Your friends with guitars and pedal boards are not one of those scenarios, however.

I think the latency overhead can be low enough these days (kernel launch / copy data from CPU to GPU, read from global memory, do some compute, write to global memory, copy data back to CPU and wait for completion from the CPU side (which might copy the data to some other buffer) or non-blocking DMA write instead), say on the order of 10-100 us over PCIe, but there is a tradeoff between the units of work that one would give the GPU, the efficiency of the compute (and the working set size of data that you need to load in from global memory to do the compute), and the number of individual kernel launches that one would need to do to produce small pieces of the output. There are some tricks involving atomics (or in CUDA in particular, cooperative thread groups) that could allow for persistent kernels that are always producing data and periodically ingesting commands from the CPU to avoid the CPU constantly needing to tell the GPU to do things to make it easier.

As far as I understand, this information is out of date.

https://www.nvidia.com/en-us/geforce/guides/nvidia-rtx-voice...

https://gpuopen.com/true-audio-next/

Neither applications are designed to run while the GPU maxed out with graphics rendering, are they?

For the unpredictable part of the input, you can't have the best latency and throughput at the same time. If you want realtime in a variable load scenario, you have to cap the GPU usage.

That's how gamers do it when they want the lowest latency possible, anyway. Something like "find the lowest frame rate your game runs on, and cap it to 80% of that".

Both those links describe audio processing on a GPU. Neither of them address (from a quick skim) the roundtrip issue that occurs when doing low latency realtime audio.

Thank-you! That explains a lot!

we have hdmi audio, is that a start? I assume that means the videocard is registering as a sound card as well, no idea if any processing is on the gpu...

100% unrelated. HDMI is an output from your video interface.

but that means the audio stream is at least passing through the video card, no? is it only passing through post mix?

the GPU is not involved in computing the audio stream. the audio side of the HDMI appears to the OS as an audio device, and the CPU generates and delivers the audio data to it. It's essentially a passthrough to the HDMI endpoint.

I would like to have the ability to use a GPU buffer for HDMI audio output. It would enable a couple of interesting applications.

Reply on news.ycombinator.com