Sanstun: A sans-IO, parallel-friendly WireGuard implementation in Rust
firezone.dev
firezone.dev
> The design also avoids problems like function coloring and plays well with Rust's ownership rules.
Async is not I/O. It can be used for I/O, but it's just a way of writing pure, deterministic state machines that keeps a logical flow together instead of scattered over a bunch of transitions.
Even if you choose to manually compile your async flows into explicit state machines, you still have to deal with ‘function colouring’ (a.k.a. typing): a flow that crosses several state transitions (and may have other disjoint parts of the state updated in the meantime, via other transitions) now has to be rewritten as a (series of) state transitions, and can't be called from a blocking function. If anything, rewriting all your state machines without `async` makes the function colouring ‘problem’ worse, because multi-transition flows are even more different from normal functions.
The problem is, the moment you use `async`, your state machine reduces down to one API: `poll`. What if I want to change some state in-between two state transitions (i.e. between two `await` points)? Now I have to wire-in a channel to supply that data at the right point or access some shared data via a Mutex.
Rust's `async` implementation is great if you know everything upfront before you kick off your state machine. The moment you need to do many things concurrently, it can become a real mess.
Firezone's sans-IO core handles many things concurrently: inbound IP, inbound UDP, WebSocket control messages, TURN allocations, timers, user commands, etc. Having essentially one big struct that I can just call `&mut self` functions on for any one of these inputs makes this very easy to understand, compose and test.
A few things like TURN request-response handshakes might be easier as an `async` function but in reality, it is not a big deal and the composability you get from it outweighs it in my opinion.
If I am now writing my own runtime / eventloop to advance my state machine anyway, what exactly am I getting from doing that through an abstraction like the Generator trait?
On top of that, the resume and yield abstraction can get convoluted because a lot of times, you only want to do one or the other, meaning either case needs to support a no-op because a single `yield` always does both.
Lastly, with all input going through a single type, your state machine then needs to handle every possible input at every yield point. That is a lot of `match` statements and branching.
I am sure there are better ways to do this and having generators to express custom iterators would be amazing. But for generic state machines, I don't see much of a benefit.
At some point maybe the ergonomics get so bad that you are just better off writing the state machine manually.
The ergonomics of this could be made reasonably friendly if your completeness checker can narrow down the set of admissible matches a la "GADTs meet their match". A single R sum type, but the compiler only makes you handle the cases that can occur at each poll() site
I think as phrased what you said is an _advantage_ of the Rust model: the state of your state machine is entirely contained within the dedicated state-machine type and you know its behaviour can't depend on surprise outside state.
But I think the spirit of your criticism is correct: you do really want to be able to type processes with intermediate inputs and outputs, in general, and Rust makes this hard (though not impossible: see Munksgaard's work on session types in Rust, for example). I would not have had a problem if the original article had decided that Rust's futures are unfit for its purposes, and it would have been an interesting case study in what exactly real-world code needs from the API; my complaint is just that they get categorized as a type of I/O, which couldn't be further from the truth.
The sans-io design here is about building reusable parts that any runtime or thread can drive and incorporate into the colored interface they expose.
Function coloring is about (the annoyance when) composing interfaces designed around those different runtimes/thread that might use the same sans-io library internally.
Yes async compiles down to a statemachine, but we're talking about the elegance of the language syntax/composition. Most languages require _a lot_ of ritual (or overhead) if you want to use the async-statemachine as a pure statemachines and sidestep the build in runtime-side-effect channels.
No, it's about the difficulty of calling async functions (or functions with other typed effects that can't be handled locally) from pure functions (or more generally functions that aren't themselves typed with that effect).
> Yes async compiles down to a statemachine, but we're talking about the elegance of the language syntax/composition. Most languages require _a lot_ of ritual (or overhead) if you want to use the async-statemachine as a pure statemachines and sidestep the build in runtime-side-effect channels.
That's true, which I would consider a limitation of those languages. Thankfully Rust futures are exactly state machines; there are some rough edges on their API still, but I'd have preferred a discussion about those to throwing the baby out with the bathwater and claiming that those state machines are a type of I/O.
I argue that it is exactly opposite. Blocking threads are a much more natural way of thinking about concurrency.
Quote from the erlang creator Joe Armstrong's thesis. He argues that the only way to understand multi-threaded programs is writing strictly sequential code for every thread and not muddling all the code for all the threads in one place:
"The structure of the program should exactly follow the structure of the problem. Each real world concurrent activity should be mapped onto exactly one concurrent process in our programming language. If there is a 1:1 mapping of the problem onto the program we say that the program is isomorphic to the problem. It is extremely important that the mapping is exactly 1:1. The reason for this is that it minimizes the conceptual gap between the problem and the solution. If this mapping is not 1:1 the program will quickly degenerate, and become difficult to understand. This degeneration is often observed when non-CO languages ["non concurrency-oriented", looking at you JavaScript!] are used to solve concurrent problems. Often the only way to get the program to work is to force several independent activities to be controlled by the same language thread or process. This leads to a inevitable loss of clarity, and makes the programs subject to complex and irreproducible interference errors." [0]
I agree that blocking threads is generally more natural / obvious. Cooperative multitasking was always done for performance reasons and more recently for simplicity (in single thread implementations with no locking) and/or scaling reasons.
Message passing concurrency is going even further though, and never really seems to have taken off wholistically. Again, primarily for performance reasons. When we do pass messages (across networks, too/from RAM or disk, etc) we usually add caching and other optimizations which make concurrency even harder to reason about.
The line is fuzzy though.
You might say that each RAM access is really a message over a bus, and so is each cache access. Conversely, you could write a message passing style key value store and create race conditions by using like shared memory.
Transactional memory is also 'message passing' like -> you bundle up access of shared memory into a single transaction 'message'.
A lot of shared memory code uses constructions like FIFO/queues which are really not too distinct from message passing except when ownership is shared or goes back and forth. Perhaps that's just an 'implementation' of message passing.
Of course if you don't have shared memory then you have to copy anyway :) And because of CPU caching architecture the benefits of having an owned copy of the data can sometimes outweigh the cost of copying it, even if shared memory is available.
We're also now talking about patterns which are common in other contexts, but I don't have a problem with that. io_uring for example - is it shared memory, asynchronous, message passing, or all three (probably yes)?
You made me curious about the spelling of 'holistic', given that it really is based on the concept of wholeness. It turns out:
- Holistic is formed from the Greek root hol-, meaning "whole". Like hologram.
- It's even closer to the English than it looks; modern whole descends from Old English hal (cognate to the Greek word), which also gives us hale and health. The W in the modern spelling is a spurious addition unrelated to the etymology of the word. You'd expect "hole".
- Who is different, and the W in that word is etymologically correct. (Old English hwa.)
> Blocking threads are a much more natural way of thinking about concurrency.
The (ideal) programming model for writing a thread vs an async task is identical, and indeed the most natural way of writing sequential flows that may incidentally be temporally separated. A blocking operation and an await are equivalent (in the absence of shared memory; in its presence they differ in that an async task on a single thread gets to assume atomicity of the code between awaits, but that's not too relevant here). I think what Joe is arguing against here is exactly splitting up your business flows into a bunch of explicit state machine transitions.
The difference is only in the caller. Callers of async tasks (à la Rust) have the right and responsibility of controlling how (and whether) the task they create resumes, while threads/BEAM processes (and ‘stackful’ futures à la Go, JavaScript, etc.) are answerable only to the great global event loop in the sky (sometimes known as the OS scheduler or the BEAM multiplexer, etc.). This makes it easier to avoid bugs that are about forgetting to drive a task, but harder to avoid bugs that are about forgetting to stop a task, or stopping a task wrongly; see structured concurrency. It also hurts composability a bit, because by construction there can be only one event loop and there's no way to decompose it.
You can see Erlang (or I suppose we should talk about BEAM more generally now people like Elixir) in terms of OS processes as an analogy, but it's just as valid and perhaps a little more accurate to think about it as a pool of async tasks with a global event loop, à la Go, where all blocking-looking code is actually async. It shares a common problem of such languages that because it can't interrupt real blocking code, e.g. FFI, you have to be very careful not to accidentally starve your other processes with such a call. This is made more obvious (but not really fixed) by languages that colour async functions differently, because the async colour acts as a promise (sadly not enforced by the language) that the function will be a good neighbour and not block for too long (whatever that means).
This is incorrect. You can easily call that flow from a blocking function, just call it in a loop.
What normally prevents a blocking function from "simply" calling an async one is that the async function needs runtime support for certain operations: the simpliest of which is suspending until it's notified, with more complex ones being async I/O primitives.
A Sans-IO library needs none of these. It can simply be called in a loop and as long as the caller provides more inputs it will make progress. An blocking caller can simply call the std's `Read::read` and pass the received bytes in, while an async caller can use async I/O primitives to do the same, and since the Sans-IO library yields once it finishes consuming the current input it won't block the async library either.
It's the best of both world but as always you have a tradeoff, and in this case it's the ability to express the state machine as if it was a straightforward function. Maybe once Rust gets support for generic coroutines this will get better.
Sure, the same can be said of an async library — so long as you're okay with none of your other flows being able to make progress in the meantime. You can just poll in a loop.
> It's the best of both world but as always you have a tradeoff, and in this case it's the ability to express the state machine as if it was a straightforward function.
‘The best of both worlds’ means that it shares the advantages of both, but the main (arguably only) advantage of async/await is that you get to write your temporally separated flows sequentially. (And maybe also that they look a bit like functions, I guess.) That's not true here.
> A Sans-IO library needs none of these.
To be clear I'm not by any means arguing that ‘sans-IO’ (or, as we used to call it, pure functional programming) is a bad thing. I'm very in favour of it. I'm just saying that a ‘sans-IO’ design shouldn't need to eschew async/await, which you can write in terms of pure functions.
(As a sibling comment points out, there are languages where that is not true, i.e. async/await inherently introduces some kind of side effect around concurrency or a global event loop. Thankfully that's not the case in Rust.)
An generic future is much harder to be called like this. You need to to create wakers and if you don't properly handle them you'll busy loop. You also need to provide the exact runtime the future needs for its I/O.
> so long as you're okay with none of your other flows being able to make progress in the meantime
And isn't that what blocking code does all the time?
> ‘The best of both worlds’ means that it shares the advantages of both
If you look at it from the perspective of a caller:
- you can easily use it from blocking code without needing a runtime - you can easily use it from async code without blocking the executor
As such it does provide the best of both worlds to the caller.
It is true that it's the worst of both worlds for who is writing the Sans-IO library though.
> or, as we used to call it, pure functional programming
You can for sure describe this as "pure" and "functional", but I bet most people think of a different paradigm when talking about pure functional programming.
You don't actually have to handle I/O with the async runtime if you don't want. You _could_ just poll the futures yourself in a loop, which is roughly the same as what happens if you just call the transition function in a loop for a manual state machine. And likewise if you just keep calling the transition function with empty input you'll also busy-loop.
> And isn't that what blocking code does all the time?
Sure, if you have only one flow you care about or you have threads to take care of your concurrency, this is fine. You can do it with any model (blocking, async, manual state machine).
> - you can easily use it from blocking code without needing a runtime - you can easily use it from async code without blocking the executor
But these things are already true of a state machine compiled from async/await instead of written by hand. `poll()` is just a function, and you can call it where you like. By the contract of `Future`, just like the contract of the state machine transition, it's guaranteed to ‘not block’. The waker machinery exists to help you write engines that decide when to poll, but if you've got your own external notion of that you're free to just call `poll()`. The standard library even has some optimization around this case, e.g. `Waker::noop()`.
> You can for sure describe this as "pure" and "functional", but I bet most people think of a different paradigm when talking about pure functional programming.
I agree, but it's confusing to me that this is the case. Original pure functional programming was all about ‘how can we write more useful programs without any effects’ and ‘what benefits can we get from doing so’. I think those questions are kind of answered; since Moggi's work in the early nineties it's been focused more on ‘how can we keep the benefits of pure functions while also being able to encode effectful logic?’, to which manually-written state machines are a rather primitive sort of answer. They bear pretty strong resemblance to iteratees/transducers/pipes/etc. from that lineage, except that those things aim to build a higher-level combinator language on top to reduce the amount of manual state-machine writing one has to do.
How would the future get the I/O result then?
> And likewise if you just keep calling the transition function with empty input you'll also busy-loop.
This makes no sense, why would I ever want to do that?
> but if you've got your own external notion of that
That's a very big if. I said "easily" and this is not easy at all.
Moreover it ignores the context: the SansIO/async/await code is generally a library written by a user, while the code that wants to use it is written by someone else entirely. Good luck sharing this external notion between the two!
---
I agree that it would be ideal to have a way to automatically generate the state machine from straightforward looking code, and async/await kinda does this, but this usecase requires more flexibility than what async/await provides.
You can write directly into the future (if you own it and know its structure), use a channel, etc. It's not the best ergonomics (see the comment uptree about this) but it's doable.
> This makes no sense, why would I ever want to do that?
Why would you ever want to wait on a Waker that you won't wake? :)
> That's a very big if. I said "easily" and this is not easy at all.
> Moreover it ignores the context: the SansIO/async/await code is generally a library written by a user, while the code that wants to use it is written by someone else entirely. Good luck sharing this external notion between the two!
It's not more difficult than with the manual state machine? That also has no general notion of inputs — you have to call specific functions to advance particular states with their inputs. There's no general equivalent of `poll`; the library and its caller have to agree on an API.
> I agree that it would be ideal to have a way to automatically generate the state machine from straightforward looking code, and async/await kinda does this, but this usecase requires more flexibility than what async/await provides.
I'm not trying to claim that (Rust's particular flavour of) async/await is suitable here. It certainly has some sharp edges, and it may well be that some of them bite for this project. I'd have loved to see an analysis of what and how, if so.
My point is just that use of async/await is not excluded by wanting to write some code ‘sans-I/O’. That's not a claim that it's universally applicable to all problems.
This is never the case for a async/await generated future.
> Why would you ever want to wait on a Waker that you won't wake? :)
Blindly waking a Waker is still the same as busy polling. You need to know whether what the async fn requested is ready or not.
> It's not more difficult than with the manual state machine?
For the end user? No. For whoever is writing the state machine? Yes. But those are generally different users.
Way more testable, the bulk can be shared between async and blocking IO, it's a lot easier to completely ban unsafe in your dependencies (excepting std and core), etc.
Another great (and subtle) thing is the monotonically increasing sequence number. Note that it isn’t incrementing, like DTLS. This small thing makes it so much easier to implement with efficient concurrency, and event external state (i.e. distributed concurrency).
That latter bit we’ve done in our own library.
Edit: typo, don’t -> done
I wonder if you've tested any related data, such as performance curves, connection jitter, and so on, as well as things like security implementations.
> Hardware does the heavy lifting.
> Your operating system and network card take over the repetitive jobs whenever they can. Your CPU gets room to breathe, and your battery stays full longer.
Where does this operating system run? Where does the power come from to run it if not battery?
Pure rubbish ai slop.
While the software might actually be ok upon inspection but this kind of ai slop nonsense copy doesn’t inspire confidence in using their software considering how security critical the vpn is.
The quoted section is meant to explain in layman's terms that we are doing hardware offloading where possible. That means vectorized instructions for encryption and checksum-offloading and packet segmentation in the kernel / hardware driver where supported.
Being more efficient per packet means that we using less power to move the same amount of data, hence less power draw on mobile platforms.
Having said that, I agree that it could be better while still not be filled with technical jargon.
You should not write things with AI that you expect people to read