Hacker News Reader

Comment by tptacek | Hacker News Reader

The general vibe I hear from security practitioners is that they (gvisor and Firecracker/Cloud-Hypervisor) are comparable. You generally do Firecracker if you can get away with it, because it's simpler and faster, but to do it well you generally need bare-metal hosts.

I think "hardware" vs. "software" is the wrong way to frame this. The big difference is between shared-kernel multitenant and guest kernels. gVisor (in some configurations) is like an 80% solution; a partially emulated guest kernel. I think the verdict is in on shared-kernel vs. not: don't share kernels between tenants.

The hardware isolation itself --- AMD-V, VT-X, &c --- has held up for almost 2 decades now. You never know, of course, but if you were going to place a bet it probably wouldn't be on hardware isolation failing before software did.

View on HN

I think this is stated confusingly.

gVisor and Firecracker are nothing alike. gVisor is an application kernel written in Go. Application kernels will make your system much slower and can only solve a subset of security issues that would affect the kernel. They're like a kernel proxy of sorts.

Firecracker is a KVM virtual machine but made much more efficient. It's the most desirable of the two. Firecracker and KVM don't require bare metal, but in a virtualized environment they will require nested virtualization support. Nested virtualization is supported by just about every hyperscaler.

The reason KVM is preferable is because it provides strong isolation of the host kernel interface and independent emulation of host devices.

I've built systems out of both and they are quite similar, using different mechanisms as vectors towards the same outcome, which is to minimize shared kernel attack surface in untrusted workloads.

Firecracker doesn't require bare metal, but (1) you want it for performance and efficiency at scale and (2) you take a security hit for nesting.

Surely the fact that one is running a "a partially emulated guest kernel" as you put it vs a real linux kernel makes them sufficiently different, no?

I haven't used gVisor in a while, but could you mount an NFS or CIFS share on gvisor or can you manage nftables, seccomp, namespaces, etc? At least that didn't use to be possible. So depending on the solution you're trying to provide, there was a big difference.

Who cares? The overwhelming majority of cloud customers just want to run straightforward Node.js apps.

In my experience, someone will eventually care, especially in enterprises or large orgs that don’t know what they are doing, then want to run dockerd or mount a share from some legacy system. Then the problem becomes do you lose that class of users or do you build 2 backends (A gVisor and a kvm/firecracker/etc) or just build a kvm one.

Google tried to do their Cloud Run gvisor only, then 2 years later switched to a hypervisor (though I don’t know if they use kvm or something else). That has been my experience also in trying to run an “emulated” Linux anything. Eventually some tough compat issue arrises and you wish you had just ran the real thing and figured all the issues with that instead. Tbf, gVisor offers much better density and much easier resource management etc. just the compat issues are tough unless you’re willing to (or can) say no to some scenarios.

I just don't think "can you run CIFS easily under it" has much to do with whether someone like Replit should use Docker, Firecracker, or gVisor.

> Who cares? The overwhelming majority of cloud customers just want to run straightforward Node.js apps.

The security professionals whose sentiments you were describing would surely care. I was answering as a software engineer that often works at this level of security. I would and do care.

Reply on news.ycombinator.com