Hacker News Reader

you trade a lot of latency to make GPU parallelism work for packet processing/classification. some massively clever work around hiding it — but simply no way to avoid it. thus its a niche solution

View on HN

With GPUDirect and active-wait kernels, you can get a tight controlled latency and saturate PCIe bandwidth without touching main memory. StorageDirect if you need to write to (or read from) disk.

packet sojurn time is bounded by the latency of the GPU memory architecture. which as I understand has the design dial cranked to ten for parallelism and not so much for expediency

People have been using GPU + DMA for low latency / real-time / high compute intensity applications for some time (using them for adaptive optics of all things). My PhD student's been cranking it to 100/200/400G, with 'just' DPDK, gpudev, and persistent cuda kernels..

Depends on the application, batching policy, compute intensity, etc. But you can put 8 NICs and 8 GPUs in one node (and have them communicate through nvlink, so huge intergpu bandwidth!) which I can't for CPUs. You can maybe also get some unobtainium A100X or CX7+H100 to skimp on PCIe if you're well funded...

They disclose their latency in the paper in section 6.4, pages 10 and 11: http://keonjang.github.io/papers/sigcomm10ps.pdf#page=10 . It's something like 0.4 ms.

Reply on news.ycombinator.com