The usual situation is like what chrome or OpenSSH want:
- They want to be able to do dangerous things by design. Chrome wants to save downloads. Chrome wants to call rendering APIs. OpenSSH wants to pop a shell.
- They want to deal with untrusted inputs. Chrome downloads things off the internet and parses them. OpenSSH has a protocol that it parses.
So you want to split your process into two with privilege separation:
- one process has zero privileges and does the parsing of untrusted inputs.
- another process has high privilege but never deals with untrusted inputs.
And then the two processes have some carefully engineered IPC protocol for talking to one another.
Could you run the deprivileged process in a VM for maximum security? Yeah that’s one way to do it. But it’s cleaner to run it as a normal process, ask the OS to sandbox it (deprivilege it), and then have a local domain socket (or whatever) that the two processes can use to communicate.
If you used a VM for deprivileging then:
- There’d be more overhead. Chrome wants to do this per origin per tab. OpenSSH wants to do it per connection. Maybe a VM is too much
- You could put the whole browser into the VM but then you’d still need something outside it for saving files. And probably for talking to the GPU. You could run OpenSSH in the VM but then that defeats the purpose (you want to use it to pop a shell after all).
- You can use vsocks and other things to communicate between host and guest but it’s much more gross than the options available when using traditional process sandboxing
Most software today relies on many (imported, third party) libraries, so the security architecture should provide primitives/abstractions to manage rights at that level, which requires programming languages to implement the ability to sandbox (managing the effects of) code. If they did this with lightweight, portable virtual machines like WebAssembly, that could work.
The vast majority of code out there should be limited to pure computation and have no ability to access anything external at all (and otherwise, only what it actually requires) - yet most languages are simply incapable of providing any such guarantees. If the programmer of software cannot get ironclad assurances, they cannot in turn provide them to their users.
I'm not saying that OS-level sandboxing isn't good, just that it doesn't go far enough. And depending on the setup, it may not sufficiently limit the effects of compromised elements, and it provides no "monitoring in the small". It's also not convenient or efficient to have an entire OS instance for every single system component. Compartmented microkernel operating systems like Genode do it better imo.