Using the IOMMU for Safe and Secure User Space Drivers [pdf]
net.in.tum.de
net.in.tum.de
The most interesting / surprising thing about this paper is how damning it is for the Intel IOMMU.
We can also confirm their result about the IOMMU TLB size of only 64 on the CPUs we have here. (Yes, the TLB size is completely undocumented and there are no performance counters...)
https://en.wikipedia.org/wiki/MINIX_3#Reliability_policies
With a monolithic kernel such as Linux, the reliability benefits of running some drivers in userspace are negligible, as the TCB is still millions of LoCs with no proofs.
Fuchsia, Genode and HelenOS are relatively active, promising open source operating systems with a microkernel, multi-server architecture and drivers running as unprivileged processes.
Try seeing it the other way around. It minimizes what needs proving (it isn't cheap) by having the microkernel (small and proven) enforce separation.
Capabilities (not to be confused with POSIX capabilities, which are something else) do help massively here. Refer to the Genode Book[0] introduction chapter for an introduction to capabilities.
In the driver example, running the driver unprivileged means the damage a faulty driver can be contained. In combination with a iommu, it can be made so that the hardware can only talk to the driver, to complete the protection. This is only possible if the kernel itself can be trusted, which would mean the kernel has to be proven, and that's unrealistic unless it's a microkernel, as it is a very costly process that doesn't even scale linearly with LoC.
The alternative is to make all drivers part of the TCB, which isn't realistic for the same reason. Unfortunately, on top of this, driver code quality is known to be particularly bad.
I see no mention of Dune [0] in the bibliography. Given the narrow focus of this research I think it is required.
Once processors are tuned for cloud workloads, they will have the affordances that will make full blown operating systems optional.
The goal we are trying to achieve in this project here is to show that drivers can (and should) be written in better languages to improve security and safety. Note that one of the drivers in the thesis is written in Rust.
And regarding XDP being safer than Rust: Yes of course. But it's also very limiting; you can't write a driver in eBPF. (It currently just prohibits jumps with negative offsets but there's some ongoing work to allow for at least some bounded loops). We are interesting in making drivers themselves safer, not applications building on top of them.
(As advisor of the thesis I however agree that XDP should have been mentioned; I'm not super happy with the length of the thesis but quite happy with the implementations)
Why? You present no evidence.
(The thesis features a performance evaluation to show that it doesn't get slower when used properly: hugepages are absolutely required; this is very different from non-IOMMU drivers where hugepages only boost performance by ~1-3% in the best case. Also, performance is simple to quantify so it makes for a great evaluation.)
Some context for this thesis/what I'm working on at the moment:
C as a programming language for drivers has failed from a security-perspective. Cutler et al. [1] analyzed 65 security bugs found in Linux in 2017 allowing for arbitrary code execution, 40 of them could have been prevented if the driver was written in a safer language. I've looked at these 40 bugs and found that 39 of them are in drivers.
I disagree with their conclusion that you should therefore consider to write the whole operating system in Go (that's just unrealistic in the near future). But we can at least start to write drivers in better languages as they seem to be the weakest point gaining 97% of the improvement for ~40% of the work. Getting a high-level language driver upstreamed in Linux is of course unrealistic, so user space drivers it is.
Network drivers are particularly interesting because user space drivers are already quite common there and there's a trend towards having network stacks in the user space anyways: QUIC is user space only, iOS runs a user space TCP stack. (Somewhat related: the future API between applications and the network stacks is TAPS instead of sockets, TAPS is also more friendly towards user space stacks as it's finally a modern abstraction for the stack.)
I agree with what you are saying, it's all true. There are lot's of problems with C in the kernel, but that's what we're currently stuck with. Practically there is only a small difference in the value proposition of rewriting the kernel in Go and rewriting our existing network drivers in userland in Rust, when our use cases are more aligned with what we're getting with XDP.
There's work being done on source-level translation from Rust code to C (mrustc is part of this of course, but nowhere near complete), so I'm not sure why this should be seen as unrealistic.
I do like the mrustc effort though.