Hacker News Reader

Comment by marginalia_nu | Hacker News Reader

Interesting. Would this still allow the virtualized machines to manage their own page tables as though native? I can't say I understand how it's implemented, but I'm concerned it might leave the host system with the same page thrashing the choice to use virtualization was intended to avoid.

View on HN

You could examine LXC, which is a container setup, for isolation. Adding memory doesn't require a restart of the container. You may wish to examine ZFS or, going farther afield, even ZFS with FreeBSD with or without jails/bhyve.

They're trying to use VMs to run multiple page table managers on one single host. Containerization will use the same kernel's page management, so I don't think that'll solve the author's problem.

I don't know if this has any impact on the page tables, but the way it works is relatively simple: the hypervisor communicates with a driver inside the VM and tells it to "allocate" large amounts of memory. How much depends on the configuration; if you tell the hypervisor to configure a minimum of 16GB and a maximum of 48GB, then the driver will "reserve" 32GB of RAM. To the VM, it's as if some kind of driver is taking up 20GB of RAM, while I reality the hypervisor just doesn't allocate the RAM to the VM.

Then, when the VM runs out of memory, the virtio driver starts releasing pages for as long as the hypervisor lets it, slowly increasing the amount of RAM allocated to the virtual machine. This works until the hypervisor runs out of memory to allocate, after which the driver inside the VM will refuse to release pages and the VM's OOM will kick in.

The default is grow only (start with the minimum amount of RAM and only release more to the VM as time goes by), but it's possible to ask the driver to start "allocating" memory again with enhancements, like the ones proxmox do or by using scripting. With the guest tools installed, the hypervisor knows exactly how much free RAM each VM has.

This shouldn't cause problems for page tables on the client side as far as I know. However, certain workloads may not work well; if the VM allocates memory faster than the balloon driver canrrelease it, you'll get out of memory errors. Similarly, quick allocate-free cycles will prevent any auto shrink mechanism from working right.

You could try and run an experiment if you wish; the balloon driver works just as well regardless of whether you're allocating 16GB of RAM or 512MB. You could run a bunch of VMs with a model of the memory allocation you expect on your desktop and see if you end up page trashing like you fear.

I don't know how VirtIO does it but VMWare has a kernel driver which allocates the pages from the guest OS.

IIRC it locks these pages so they are unavailable until the driver releases them back.

that depends on your setup. ballooning is just a function to move memory from the guest to the host; if you move too much memory you'll get thrashing. the bigger problem is I/O indirection through the host; for I/O heavy workloads you'll want to use PCI passthrough for best performance, which (assuming it's working properly) should be identical to native performance.

Reply on news.ycombinator.com