Hacker News Reader

Comment by anderspitman | Hacker News Reader

Usermode WireGuard would be a big deal. I maintain a list[0] of tunneling solutions, and one of the only limitations of systems built on WireGuard is the requirement for admin privileges. Even with the performance hit from running outside the kernel, UDP-based tunnels have a lot of advantages for multiplexing channels. Pretty much your only mainstream options today are QUIC and WireGuard, and only QUIC is intended to run in userspace.

I'd have to dig into the details more, but something like this might allow you to implement a simple tunneling system based on WireGuard that runs in the kernel if you have the privileges, otherwise falls back to usermode and is no worse than QUIC in terms of performance. That would be awesome.

[0]: https://github.com/anderspitman/awesome-tunneling

View on HN

Very cool!

Found a gap, Linux Foundation's FD.io's VPP (a high performance network virtual switch) has native wireguard support as well, all in userspace. Support here means you can do full kernel bypass from the app all the way down to the NIC card (e.g. via DPDK).

https://docs.fd.io/vpp/20.09/d5/d54/wireguard_plugin_doc.htm...

I'll open a PR on this later.

> Pretty much your only mainstream options today are QUIC and WireGuard, and only QUIC is intended to run in userspace.

Not sure it fits your "mainstream" qualification, but many projects ago I used Airhook to help create a userspace, application-layer('ish), multi-channel virtual network: http://airhook.ofb.net/ (https://github.com/egnor/airhook)

Airhook is a relatively low-level library that handles framing and flow control; it's not a functional solution on its own. But that seems to be what you're getting at--something you can deeply integrate into your application, not a separate service. Though, I guess containers have sort of muddied that distinction.

It sounds like we could have a generic userspace tool that proxies any connection to a WireGuard server. Similar to ssh -L, it would listen on a TCP/UDP port locally (or talk the SOCKS protocol) and convert that to IP packets over the WireGuard connection (using a userspace TCP or UDP implementation for that side).

It looks like Fly.io has all the bits, they just need to be packaged as a stand-alone tool rather than built into flyctl and only talk SSH.

Tailscale will do this!

    tailscaled --tun=userspace-networking --socks5-server=localhost:1081

Sadly, on the only machine that I would have wanted this on, where I didn't have root access, this has never worked for me.

I should try to recreate the logs and issue for the tailscale folks.

I had issues with it as well, was following the Tailscale guide for getting userspace running in Azure App Service and I could not get it to work.

Our customers at Render run Tailscale in user mode every day. Here's the repo they use: https://github.com/render-examples/tailscale.

An example of using Tailscale to access VSCode in the cloud: https://render.com/blog/host-a-dev-environment-on-render-wit...

With any WireGuard server or a custom tailscale server/service?

SSH can do this without any WireGuard:

https://rkeene.org/viewer/tmp/ssh-ip-tunnel.txt.htm

There's nothing userspace about this, you are using a tun device with the actual kernel IP stack.

One thing to watch out for with that setup is using SSH for TUN devices can suffer from TCP-over-TCP performance issues, aka "TCP meltdown", when there's packet loss. You can avoid this by using normal SSH tunnels, a la ssh -L or -R, which unpack the individual TCP streams and multiplex them over a single connection. Or if you need a more traditional VPN setup use WireGuard.

Is https://tailscale.com/ not "usermode WireGuard"? I've been playing with it for a while now (it has a fairly generous free tier) and am quite impressed. I can access any of my LAN machines (my servers, my NAS, etc.) from anywhere that is also connected to the same network, and the names work for DNS as well.

1. Tailscale is amazing. I hate them so much. (We use Tailscale and are very happy with it.)

2. Tailscale is user-mode WireGuard.

3. "User-mode WireGuard" in the sense this post uses the term is a misnomer and refers to the fact that we run TCP/IP itself in userland (Tailscale normally runs through a tunnel device and uses your native TCP/IP stack).

4. But Tailscale also has code to do user-mode TCP/IP (they've got it running in a browser with wasm).

Fascinating!

Last I heard[0] they were experimenting but hadn't shipped it. AFAIK their client still requires root, no?

Running on wasm sounds awesome. This[1] looks like it. Do you know how they're doing the actual networking? WebRTC tunnel?

[0]: https://news.ycombinator.com/item?id=24483173

[1]: https://twitter.com/bradfitz/status/1451423386777751561?lang...

Yeah, their client is always going to require privileges, because it needs to enable every other program on the system to interact directly with remote hosts transparently. User-mode TCP/IP works for us because we own the client-side program that our users run to talk to stuff on Fly.io.

> Last I heard[0] they were experimenting but hadn't shipped it. AFAIK their client still requires root, no?

Tailscale's gvisor/netstack-based userspace networking mode has been supported and in wide use for quite some time. It's the default on Synology DSM7, for instance.

You don't need root when you run tailscaled with `--tun=userspace-networking`.

Peers can still connect inbound to the non-root tailscaled, but to connect _out_ to other peers, you need to use tailscaled's HTTP or SOCKS5 proxy, which are also flags to tailscaled, to specify what port they listen on.

Thanks for the update!

Do you have any links that talk more about how the wasm stuff works? I'd love to read more about that.

I think Tailscale uses user-mode TCP/IP (also gVisor netstack) for some client devices, like iOS? But could be wrong here.

We use it on all platforms _except_ iOS, for binary size/memory reasons.

(iOS 15 bumped the Network Extension memory limit to 50 MB, but we still need to be super trim for iOS 14's 15 MB limit)

LOL, I was precisely wrong.

Is there actually a preference for user-mode networking? I assume that’s primarily about control and flexibility?

Either way, I hope that the PacketBuffer changes can help reduce footprint after issues are shaken out.

It depends on what sort of tunneling you're doing. If you just want a general-purpose private VPN, Tailscale is amazing. That list is more focused on the use case where you want to host a public server on a machine that isn't accessible to the internet (NAT, corporate firewall, etc). Think a shared Jellyfin server for your friends and family.

You can use Tailscale here but you'll need to separately run a reverse-proxy on a public machine. There are more moving pieces but if you're using Tailscale already then it's a good option.

I wish I could run two separate Tailscale networks on a single device, one for business and one for personal (for example). Would make it tremendously more useful.

There's an existing GitHub ask for this to be implemented. It's not terrible jumping between work and personal and work environments but it would be nice if I didn't have to.

Is this different from sharing one device with other accounts?

Headscale seems even better! They've taken what tailscale has done and improved it even more by allowing it to be a completely self hosted and private solution.

TunSafe also runs on userspace: https://github.com/TunSafe/TunSafe

It looks like from the source code that TunSafe opens up a tunnel device, in which case it's doing TCP/IP in the kernel, not in userland.

It opens tun/tap device just to get/push packets from/to virtual interface. Using tuntap is very common for all VPNs like OpenVpn2.

There isn't kernel driver like in real WireGuard.

Reply on news.ycombinator.com