Hacker News Reader

Comment by comboy | Hacker News Reader

Works fine on opnsense.

But I'm pretty sure I've seen wireguard settings on pfsense a few months ago (non-commercial version)? So what is new here?

Btw, I'm wondering what HN folks think but I'm leaning to basically go full wireguard. I don't see why I wouldn't want it basically everywhere. It simplifies network management, makes things more explicit. There don't see to be any performance issues (running production database through it right now), and when running it at home it can even make my Internet seem faster when tunneling through it (probably because of some suboptimal TCP settings somewhere along the way, which seems common).

View on HN

kernel vs user-space implemention.

Oh, and why would you want to have it as part of the kernel?

As the other poster said, performance. The idea behind wireguard was that it would be a VPN protocol small enough to implement securely in kernel space.

Performance:

https://www.netgate.com/blog/wireguard-in-pfsense-2-5-perfor...

DPDK is fast in userspace by taking ownership of devices (I believe) so that there's no context switching anyways.

My understanding is DPDK has much smaller set a supported nic's since it doesn't use the kernel drivers for the card along with all the other things that the kernel provides.

Wireguard in kernel gives great performance while just working if your kernel is already working on the hardware. Wireguard is so small it is considered easy to audit so is a prime candidate for kernel implementation.

DPDK is listed as an "Exotic" todo for Wireguard:

https://www.wireguard.com/todo/

There is already an implementation of Wireguard in VPP, which runs over DPDK.

Throwing out a few points regarding DPDK/low overhead data/event exchange between kernel and userspace:

1. It is faster if you make it so, i.e., you need to invest quite some work to get to a point where it's faster than kernel - but yea, with that you can make it go brrrrrrrt in quite a few use cases, it's more like an SDK.

2. Some NICs support loading a smaller program to operate directly on the NIC, e.g., through eBPF - nothing gets faster than that as it avoids that (some) traffic hits the CPU at all. But again, it's work to create and maintain and only feasible for specialized stuff.

3. Note that with Linux[0] a lot of the context switch overhead can be avoided by leveraging the state of the art async framework IO-uring, BSD may already have or get something similarily[0]. I recommend reading the paper of the original author[0], it's fascinating IMO.

[0]: I know this is about BSD, so somewhat off-topic, but still shows that most of the context switch complexity can be avoided while not re-inventing the wheels in user-space.

[1]: https://kernel.dk/io_uring.pdf

I'll add that with DPDK you end up needing your own TCP stack if you're doing anything above layer 3.

TCP is a beast with opportunity for countless subtle bugs. The Linux kernel is perhaps the best implementation there is. Many userspace implementations are simplified and missing features.

Even giants like Cloudflare use Linux kernel for routing when they need to operate above layer 3. They only use DPDK for very low level features.

DPDK has its place but for most use cases you need to man handle TCP and you're better off using kernel packet handling for that

> use Linux kernel for routing when they need to operate above layer 3.

What could you possibly mean by this?

When they need to handle UPD and TCP. Cloudflare's web application firewalls run at TCP and HTTP layer and use tuned Linux kernels for processing, not DPDK and userspace TCP as some think

Their line rate stuff for flood mitigation uses DPDK or similar

Nothing there was DPDK, just FreeBSD kernel networking.

The Wireguard implementation in VPP runs at about 5Gbps in a simple single-stream iperf test.

5Gbps on what hardware and how does that compare to the kernel implementation on the same hardware? Curious to know how big a difference DPDK makes. Also what about cpu time, I know typically DPDK takes more CPU due to polling, normally want to dedicate cpus to DPDK right?

Very new Xeon at 3.5GHz. 5.75gbps.

Kernel Wireguard is almost 3gbps.

But Wireguard is mostly held back by the slow crypto implementation, which is CPU-bound.

AES-GCM IPsec on the same hardware is 19.5gbps, limited by the 25gbps NIC, using again, single-stream iperf with ipsec in VPP.

DPDK only needs dedicated cores for the poll mode drivers, and these days the Intel and mellanox NICs can be run in interrupt mode.

I thought interrupt mode reduced performance, that was the tradeoff, is the throughput your quoting in interrupt or polling mode?

Good numbers either way, but you again much more limited hardware support since the drivers need to be DPDK specific.

You can also run VPP over AF_PACKET (slower, but faster than the kernel) of AF_XDP (almost as fast as DPDK interrupt mode.)

But Wireguard is still crypto-limited, the AES-GCM result proves it.

Wireguard isn't doing to bad considering its not hardware accelerated like AES-GCM.

Also looks like AF_PACKET and AF_XDP are basically an alternative to DPDK with similar functionality but built into the kernel using kernel drivers, new to me.

I've given up on Xsense distros and use OpenWrt. It's Linux based so Wiregaurd is in kernel and easy to setup.

I keep trying to get people to try it out :) . I left pfSense community years ago when drama started and it became clear somebody was commercializing it.

Linux based distros have better features and hardware compatibility these days anyways. BSD is slowly dying (at risk of downvotes here)

Even Donenfeld has said the Userspace implementation in Go isn’t very good.

Is it even 1.0 yet...?

Reply on news.ycombinator.com