This thread has a good answer: https://news.ycombinator.com/item?id=16976392
> "containers do not contain"
Is sort of troll bait. They do contain. That is why everyone is using them. Sure, there will be exploits to break out of them, just like with VMs, and even CPU bugs now.
Here is a good example of someone who broke out of a container on the play-with-docker.com site using a custom kernel module [1]. This allowed a container escape but you could say this was a bug since that wasn't the intent. So, you'd patch it. So, I get the joke in that people are extremely creative and will find ways around everything.
[1] https://www.cyberark.com/threat-research-blog/how-i-hacked-p...
That one was an extremely obvious misconfiguration - running with --privileged=true. There dozens of ways to abuse that, probably much easier than using a custom kernel module.
Yes, containers do contain, but the attack surface is MUCH larger than a virtual machine or something like gVisor. Just look at the constant stream of Linux local privilege escalations.
"people are extremely creative and will find ways around everything" is not an excuse - it's a matter of risk management and threat modelling.
Escaping from a VM or gVisor is much, much harder than escaping from Linux namespaces ("containers") due to the MUCH smaller amount of attack surface/amount of exposed code. Using Linux containers in an untrusted multi-tenant environment is very dangerous, especially if you're a high profile cloud provider, which is why all of these projects exist.
Their breach isn't really a good indicator as I can't think of any/many reasons that most companies would try and do that...
There are a bunch of legitimate reasons to run Docker in Docker. The most obvious is in a build pipeline. For example Jenkins does Docker builds in containers all the time.
You can't get that from normal Linux containers due to their fundamental design.
At what point are we just reinventing the VM hypervisor, but worse because every single one of these systems already has a VM hypervisor running somewhere? It seems likely to me that in the not-so-distant future the "Container" terminology won't actually mean anything because we'll figure out the engineering difficulty behind merging the best parts of VMs with the best parts of Containers, and managed systems like Fargate or even GKE don't really need both a VM hypervisor and a Container hypervisor when they're so similar.
The main difficulties with VM-backend containers are storage passthrough and memory overcommit.
For storage, there's already virtio block devices, not to mention PCI passthrough. But if you mean direct file system access, virtio-fs (https://virtio-fs.gitlab.io/) is just about ready to roll.
There's still the issue that you're running an entire extra kernel. Not sure that's much slower than using Go; it's probably faster if what was described about bouncing on futexes elsethread is true.
gVisor sounds like the kind of solution that makes sense for Google but not something that would survive in the wider community. The concept sounds great, but using Go sounds horrible, though I'm sure Go made prototyping the concept super simple--specifically goroutines reify execution flow in a nice way, but so would stackful coroutines in C or even Rust, which is easy to implement if you don't need to worry about deep recursion.
As it kind of proves a point about systems level software being written in Go.
I would rather see VM/unikernels take off instead.
[1] The original fix was to exec runc from a memfd-backed copy so writing to /proc/self/exe in the container didn't poison the binary outside the container. But the change in memory usage broke some existing workloads in the wild which had low memory resource limits. I think the second iteration used O_TMPFILE on tmpfs, or at least that was what was under discussion.
The non-security related issues you listed, specialized interfaces to allow I/O to bypass the generic hardware virtualization layer, are IMO hacks (even the name of "para-virtualization" given to such mechanisms should be a tell). Because it would be to inefficient to do almost any I/O we care about to perform fast (network and storage) through the overall machine virtualization interface, we poke holes in that interface, specialized ones, that will allow us to carry requests and replies from the guest to the host more directly/efficiently. As a software engineer that seems like a hack. When something like gVisor comes along which provides much better security for the host environment and allows to quickly handle syscall level I/O by design I much prefer that approach over a VM. The drawback of gVisor is one similar to Wine: having to write bug for bug compatibility with the ABI supported (Linux x64 in this case). However, different from Wine, the Linux ABI surface is extremely small vs to what Wine has to reimplement to run even the simplest Windows applications and, most of all, with gVisor there's direct access to the source code of the ABI that it needs to implement making development much easier than something like Wine.
https://gvisor.dev/docs/architecture_guide/ > gVisor is written in Go in order to avoid security pitfalls that can plague kernels. With Go, there are strong types, built-in bounds checks, no uninitialized variables, no use-after-free, no stack overflow, and a built-in race detector. (The use of Go has its challenges too, and isn’t free.)
Using a memory-safe language was a conscious design decision. https://twitter.com/LazyFishBarrel/status/112900096574140416...