Hacker News Reader

Comment by ppoint | Hacker News Reader

Interesting read, thank you!

Have you considered moving into userland? E.g DPDK/netmap/etc + BSD stack extracted?

A team from Cambridge wrote an interesting paper for SIGCOMM 2017 "Disk|Crypt|Net: rethinking the stack for high performance video streaming" The main gain they get out of netmap, etc, is tighter latencies which allows them to make more effective use of DDIO (I/O caching in L3) to free up memory bandwidth. However, their results are on synthetic workloads, and the general feeling is that we would not see nearly as much of a benefit in the "real world".

Interesting. Usually moving networking operations into userland allows your application to "own" the NICs and the stack and reduce lock contention in the packet path down to virtually zero.

My impression was that profiling userland applications is easier too, but I haven't done any serious kernel profiling so I might be wrong.

The hardest part is, of course, ripping the stack out and keeping it up to date with the mainline kernel afterwards if you need TCP.

There is a port of the FreeBSD network stack called libuinet so that's not an unsolved problem, but it would need a bit more care and feeding. There are other issues like.. you need to fetch content off disks, and then you need to move it between address spaces, and then you need buffer types to pass around and then and then.. at some point you are reinventing a lot of wheels for the same goal: saturate some hardware limit like CPU, memory bandwidth, bus bandwidth, drive bandwidth and latency. The FreeBSD kernel works well for all this and is proven.

Userspace networking is a really big win for packet processing which doesn't have any of the above concerns. On FreeBSD with Netmap and VALE you can chain things together in really interesting ways where you can still use kernel networking where advantageous and userland networking where it's advantageous.

> general feeling is that we would not see nearly as much of a benefit in the "real world".

Wouldn't any benefit be interesting? Or is the engineering overhead just too expensive?

Reply on news.ycombinator.com