> With this new controller, it’s possible to gain anywhere from 35-42% better memory usage in Linux.
Considering the gain I'm surprised there wasn't more discussion or interest in this.
Considering the gain I'm surprised there wasn't more discussion or interest in this.
~10% of the utilization on my system is slab.
[bjames@lwks1 ~]$ grep "Slab\|MemTotal\|Active:" /proc/meminfo
MemTotal: 65976620 kB
Active: 6139820 kB
Slab: 605988 kB
That's hardly insignificant.
It sounds to me like slab allocation is specifically used to allocate small blocks of memory (less than a page). Ie a slab might be set aside for integers, then if an application (edit: not applications, see rayiner's comment below) needs to store an integer it gets stored in that slab instead of somewhere else in memory. I'm not a Linux developer, so I hope I'm not spreading bad info. Maybe someone else can chime in.
I assume this would largely be short lived allocations so I can see where optimizations here would lead to power savings.
¹ On my machine it's currently at only about 165MB, so not quite as extreme, but still a decent chunk of memory.
To the degree that you can achieve "getting the kernel to do your work for you", the kernel memory allocator becomes one of the main determinants on your scaling requirements. IIRC WhatsApp hit this point with the FreeBSD kernel, and had to tune the heck out of the kernel to keep scaling.
(Tangent: why is it that we hear a lot about bespoke unikernels, and a lot about entirely-in-userspace designs like Snabb, but nobody talks about entirely-in-kernelspace designs? Linux itself makes a pretty good "unikernel framework": just write your business logic as a kernel driver (in e.g. Rust), compile it into a monolithic kernel, and then run no userland whatsoever.)
But, you can always write a five-line "trivial init" in C that just starts and then goes to sleep forever. If you didn't know, init processes don't have any special kernel-imposed requirements. You can run /bin/bash as init; this is what many distros' single-user rescue modes do!
Or, if you're even lazier than that, I believe Busybox has a configuration wizard that allows you to just deselect all the features. That'd probably spit out a "trivial init" binary.
I disagree on that one. More memory in the Kernel mean more chances for going OOM, more fragmentation of kernel memory, and less isolation and stability.
In the ideal case I would rather want the least possible amount of memory in the Kernel (and maybe have that all statically allocated) in order to maximize stability and determinism.
And does “fragmenting kernel memory” mean anything, if you preallocate a memory arena in your kernel driver (taking “everything the base kernel isn’t using”, like a VM memory-balloon driver), and then plop a library like jemalloc into the kernel to turn the arena into your driver’s shiny new in-kernel heap? You’re not messing with the kernel’s own allocation tables, any more than a KVM VM’s TLB interfere’s with the dom0 kernel’s TLB.
See also: capability-based operating systems (like Microsoft’s ill-fated Midori), where there’s no such thing as a per-process address space, just one big shared heap where processes can touch any physical memory, but only through handles they own and cannot forge. If your OS has exactly one process anyway, you don’t even need the capabilities. (This also being how managed-runtime unikernels like Erlang-on-Xen work.)
Also, another example of what I’m talking about re: benefits of this approach. would you rather that a VM operating as an iSCSI disk server ran as a userland iSCSI daemon managing kernel block devices; or—as it is currently—as an entirely in-kernel daemon that can manage and serve block devices with no context switches or memory copies required?
Running everything (or as much as possible) in Kernel mode has obvious performance benefits, hard to argue against that.
The counter-argument isn't about performance, it's about flexibility. As far as I'm aware, no major cloud providers allow you to run apps in Kernel mode. So if you develop an app on your own hardware, but may eventually deploy it in the cloud, you better put everything in user mode. If you want to switch from one cloud provider to another, it's (relatively) easy if everything is in kernel mode.
Also, while running in Kernel mode is almost certainly faster than user mode, it's probably not that much faster. If your app relies on the network or is heavy on disk I/O, that's where your bottleneck will be, not OS user/kenel mode switching.
In short, running things in Kernel mode may sometimes be a good performance decision, but it's often a bad business decision.
I’m not familiar with this particular deployment process, so don’t want to speak out of line... but this being the internet, why not.
I’m a bit skeptical that Amazon would let you run anything in pure kernel mode, it’s likely a VM/ sandbox wrapping an OS that’s operating in Kernel mode, likely negating much of the performance benefits.
Second, you mentioned two specific images, and I’ll assume they work fine on AWS, but they are just two, and if your working with them, you probably have very specific needs, not suitable for general development.
Third, who knows if these images will work on other cloud providers. Once you get your kernel mode app working on one, your locked in.
Fourth, what are you doing that requires this level of local machine performance operating in the cloud? It’s probably almost always better to invest your optimization time/dollers elsewhere.
Fifth, if this was a good / easy idea, many people would be doing it, but they aren’t. Either you’ve stumbled upon some secret enlightened approach, or your probably wron.
And it's quite possible to create new images for these instances for your needs.
You're right about the complexity and lock in. Using such services are often a trade off in dev complexity vs speed. If you're following a well defined process that doesn't change often that's going to peg the CPU of a 72-core machine for days, it might be worth it to eek out every speed improvement you can. If, on the other hand, you're constantly iterating and updating your code, and responding to handful of user requests that barely causes a blip on a single core VM, CPU speed is probably not your primary concern.
People are doing it, I'm one of them. It's a niche though, and most assuredly not for ever case.
Writing your code to be run as the kernel of the VM, on the other hand, reduces this to one context switch and one page translation, as your application is just making hypercalls directly and directly using "physical memory" (≣ dom0 virtual memory.)
Think of it this way: from the hypervisor's perspective, its VMs are a lot like processes. A hypervisor offers its VMs all the same benefits of stability and isolation that an OS offers processes. In fact, the only reason they aren't just regular OS processes (containers, essentially), is that IaaS compute has been set up with the expectation that users will want to run complete boot-images of existing OSes as their "process", and so a process ABI (the hypercall ABI) is exposed that makes this work.
But, if you are already getting the stability+isolation benefits just from how the IaaS compute provider's hypervisor is managing your VM-as-workload—then why would you add any more layers? You've already got the right abstraction! A kernel written against a hypercall interface, is effectively equivalent to a userland process of the hypervisor, just one written against a strange syscall ABI (the hypercall ABI.)
(And, of course, it's not like you can choose to run directly as a host OS userland process instead. IaaS compute providers don't bother to provide such a service, for several reasons‡.)
> Third, who knows if these images will work on other cloud providers.
Hypercall ABIs are part of the "target architecture" of a compiler. You don't have to take one into account in your source code; compilers handle this for you. You just tell clang or ocamlcc or rustc or whatever else that you're targeting "the Xen hypercall ABI", or "the ESXi ABI", and it spits out a binary that'll run on that type of hypervisor.
(Admittedly, it's a bit obtuse to figure out which hypervisor a given cloud provider is using for a given instance-type; they don't tend to put this in their marketing materials. But it's pretty common knowledge floating around the internet, and there are only four-or-so major hypervisors everyone uses anyway.)
> Fifth, if this was a good / easy idea, many people would be doing it, but they aren’t.
I'm from a vertical where this is common (HFT.) I'm just here trying to educate you.
---
† there are in fact "bare-metal clouds", which do let you deploy code directly on ring 0 of the host CPU, with the same "rent by the second" model of regular IaaS compute. (They accomplish this by relying on the server's BMC—ring -1!—to provide IaaS lifecycle functions like wiping/deploying images to boot disks.) It's on these providers where a Linux-kernel-based (or other FOSS-kernel-based) unikernel approach would shine, actually, as you would need specialized drivers for this hardware that Linux has and the "unikernel frameworks" don't. See http://rumpkernel.org/ for a solution targeting exactly this use-case, using NetBSD's kernel.
‡ Okay, this is a white lie. Up until recently none of the big IaaS providers wanted to provide such a service, because they didn't trust container-based virtualization technology to provide enough isolation. Google built gVisor to increase that isolation, though, and so you can run "ring-3 process on shared direct-metal host" workloads on their App Engine, Cloud Functions, and Cloud Run services. But even then, gVisor—despite avoiding ring-0 context switches—still has a lot of overhead from the user's perspective, almost equivalent to that of a ring-0 application in a VM. The only benefits come from lowered per-workload book-keeping overhead on the host side, meaning Google can overprovision more workloads per host, meaning that "vCPU hours" are cheaper on these services.
Thanks for the clarification, this does indeed make sense. If your app is already sandboxed by the VM, introducing a second kernel/userland sandbox within the existing sandbox doesn't make as much sense.
That said, I think there are better ways to fix this issue than putting all of your code into a VM's kernel space. For instance, imagine there was a way for a hypervisor to lock down and "trust" the code running in a VM's kernel space, and could thus put the VM's kernel space into the same address space as the hypervisor. This could also potentially reduce the two memory translations down to one.
Another solution is to rely more on special hypervisor hardware that could conceivably do the two memory translations (VM user -> VM kernel -> hypervisor) as fast as a single translation.
The main reason that these alternative approaches may be desirable, is that asking developers to move their programs from userland to the kernel is a big ask. There's a lot of configuration that needs to be done, and few general software developers have experience working within unprotected kernel space. Simple bugs that would normally just crash a single process could bring down the entire VM, and could potentially affect other VMs on a network (for example, imagine a bug that accidentally overwrote a network driver's memory).
I'm sure there are performance gains to be had here, but they may be insignificant. Projects like these are cool, but raise big red flags of potential over and early optimization.
A while back, I worked for an embedded systems company that developed real-time OSes, compilers, and debuggers. Everything in the Kernel had to be statically allocated, malloc was simply not allowed. Even user-land programs, which could dynamically allocate memory, had to check the return value of malloc to make sure the memory was actually allocated (malloc returns null if it can't allocate memory).
It was painful to program in this environment, but over time, I grew to love it. Software developers did not feel like they were developing "Soft"-ware; memory, CPU, I/O, interrupts, and even heat dissipation were not infinite resources you could use without care. These restrictions elevated the development process, and made the "developers" feel more like "engineers".