Hacker News Reader

Comment by notjack | Hacker News Reader

This assumes that Docker containers are being used like VMs. They're not designed to allow running isolated arbitrary code in some sort of multi-tenancy setup, they're designed to isolate dependencies and configuration between your own deployed services. This is a "security vulnerability" in the same way that putting up a white picket fence around a jail is - a gross mis-application of a tool built for a completely different purpose.

View on HN

Listening to what the Docker people write about security, they sure do sound like they are designed to be secure, even though they admit to a couple of potential shortcomings versus VMs.

https://blog.docker.com/2013/08/containers-docker-how-secure... concludes,

"Docker containers are, by default, quite secure; especially if you take care of running your processes inside the containers as non-privileged users (i.e. non root)."

They also recommend using SELinux with Docker to beef up security (see https://blog.docker.com/2014/07/new-dockercon-video-docker-s...)

As long as people are using containers as a security boundary it makes sense to pay attention to things like this one about the control socket.

There is a section in that link titled "Specific Attack Surface of the Docker Daemon". Meanwhile they are ignoring the attack surface of the Linux kernel. There is no mention of seccomp in the whole post.

People find privilege escalation bugs in the Linux kernel often enough that you can't really claim that running arbitrary native code is "secure" unless you're doing something to mitigate these. Things like:

http://www.openwall.com/lists/oss-security/2015/07/22/7

Note that SELinux won't do anything about this kind of bug. You have to use seccomp and similar to disable large swaths of the Linux kernel API, particularly exotic parts that aren't well-tested or reviewed.

there is no mention of seccomp because virtually no software can utilize it.

As a Chrome, systemd, and Sandstorm user, I'm a little curious what you mean.

Sandstorm.io (disclosure: my project) applies mandatory seccomp and other attack surface reduction yet manages to run a lot of apps with only minimal modification:

https://docs.sandstorm.io/en/latest/developing/security-prac...

Admittedly, though, "only minimal modification" is probably too much modification for Docker's goals. That's fair. But I don't think that gives them a pass to pretend the problem doesn't exist.

qemu uses seccomp. If you run your VMs with libvirt, then seccomp can be used to protect the host from qemu easily and routinely.

OpenSSH and BIND use seccomp. :)

I second the idea of using SElinux to secure docker because most people really are not able to fully deal with the complexity of SElinux. If they would understand security they wouldn't be jumping onto the docker hype bandwagon in the first place.

To paraphrase Theo de Raadt:

‟You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can’t write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes.”

[1] http://blog.valbonne-consulting.com/2015/04/14/as-a-goat-im-...

> They're not designed to allow running isolated arbitrary code in some sort of multi-tenancy setup

You mean Linux containers are not designed for this, nor are they designed to be secure. What a sad design failure.

Some [1] container technology was developed with security as a first principle.

[1]: http://us-east.manta.joyent.com/jmc/public/opensolaris/ARChi...

I should clarify I mean Docker containers specifically, good point.

And you can run Docker containers on SmartOS, using "LX-branded zones", which do Linux syscall emulation:

https://www.joyent.com/blog/triton-docker-and-the-best-of-al...

I'd be curious if the Joyent folks believe Triton to have the security properties that everyone seems to want out of Docker. It doesn't involve the standard Docker daemon, so it might.

@geofft that is a great link, our CTO Bryan Cantrill continued to beat the drum at the Docker SF meetup in May 6, 2015. Video and transcript at https://www.joyent.com/developers/videos/bryan-cantrill-virt... . My favorite line is, "And, so in terms of the security of containers, from our perspective they have to be secure because if you're able to break out of a container into the global zone of SmartOS, you could delete the company."

Yesterday at http://containersummit.io/, Bryan's talk "Going Container Native" (video coming soon) was in the same vein, SmartOS (illumos) directly benefits from its Solaris lineage -- Solaris Zones were engineered for security. We, Joyent, absolutely do believe Triton has the security properties that everyone seems to want out of Docker, because Joyent has been running OS containers in multi-tenant production since ~2006.

Open source SmartDataCenter [1], rebranded Triton, is the exact same code that is run in Joyent's Public Cloud. This truth in code, product and service contributed to me leaving an OpenStack company to join Joyent two years ago.

1. https://github.com/joyent/sdc

Bryan's talk "Going Container Native" from earlier this week is now online: http://containersummit.io/events/sf-2015/videos/going-contai...

As far as I can tell, Solaris Zones are no more secure than Linux containers; both use kernel-level isolation mechanisms, while sharing the same kernel between zones/containers. And thus they both have the same security property: secure as long as you can't successfully exploit a syscall, but if you can, you get kernel-level access.

The security property of "secure unless there's an exploit" applies to basically everything (maybe excluding formally verified software), including Xen and KVM, so I don't see how that's a specific knock against Zones. Yes, if there's an exploit that lets you do something you shouldn't be able to do, you can break out of the container/VM. But, like Xen and KVM, Zones are so far fairly successfully used as an isolation mechanism for the public cloud, for some years now. And all three make the claim that you ought to be able to rely on their isolation mechanisms, and any hole in them will be considered a serious bug. Whereas the way Docker is using Linux containers doesn't seem secure enough to make that claim or be used for that purpose, at least not yet.

> Whereas the way Docker is using Linux containers doesn't seem secure enough...

Which is totally expected, because docker tries to be useful to the largest user-base possible. It would be quite harder to use if didn't support directory mounting (via -v). And it would be a total nightmare for almost every user if you had to specify a list of allowed syscalls for every container.

This reminds me the situation with SELinux a lot. It has improved a lot but I still see "disable SELinux" almost in every tutorial I read on CentOS, Fedora or RHEL.

> This reminds me the situation with SELinux a lot. It has improved a lot but I still see "disable SELinux" almost in every tutorial I read on CentOS, Fedora or RHEL.

Because security is hard.

Just look at the Mac OS X users running as root and disabling Gatekeeper.

Or the developers that stay away from the sandbox model.

One of the nice things of mobile OSes is that there isn't a way around the container model. Although the history with permissions kind of messes it.

> The security property of "secure unless there's an exploit" applies to basically everything (maybe excluding formally verified software), including Xen and KVM, so I don't see how that's a specific knock against Zones.

Defense in depth. With KVM, you'd have to exploit the guest kernel, then exploit KVM's hardware emulation to get to host userspace, then escalate to host root/kernel. (And KVM can use seccomp to limit itself, making that last step harder.) With Linux containers or Solaris Zones, you run directly on top of the host kernel, so you just need a single kernel exploit.

In any case, Zones and containers have the same security properties (running directly on top of the kernel), so the original comment about Zones being more secure than containers makes no sense.

I know of only one Solaris Zone escape CVE in its entire existence.

http://www.cvedetails.com/cve/CVE-2008-5689/

I also know of only one FreeBSD Jail escape CVE in its entire existence.

https://www.freebsd.org/security/advisories/FreeBSD-SA-04:03...

These were years ago. Jails and zones are everywhere. You're kidding yourself if you think people haven't tried to attack them.

Oracle doesn't tend to disclose security vulnerabilities; they just silently fix them; the absence of CVEs is not the absence of vulnerabilities. On top of that, a vulnerability wouldn't need to be Zones-specific; almost any kernel exploit would work, unless it uses a syscall that only works in the "global" zone.

You forget that Sun was in charge of Solaris for many years and was pretty open about such things.

Any exploit against the kernel gaining code execution is a Solaris Zone escape or Freebsd jail escape.

Solaris got hit by the one in 2012 which is the non-canonical RIP which hit a bunch of operating systems.

FreeBSD has had numerous kernel exploits which could be used for zone escapes.

Really? Which ones? I'm pretty sure local privilege exploits that give you root don't magically jump you into JID 0 (prison0), escaping the jail. You'd just have "root" in the jail.

edit: well i guess if your attacker knows you're on FreeBSD they could override to prison0 in the exploit. But it's a good thing we dont have a ton of local priv exploits, eh?

edit2: and that was a pretty ignorant statement of mine about root exploits anyway :)

Linux containers are certainly intended to be secure. The problem is that the Linux kernel API is huge and privilege escalation bugs are found all the time. In order to make a container secure, you need to disable 95% of this API, e.g. using seccomp, not mounting /proc or /sys, drastically limiting /dev, etc. Sandstorm.io hasn't seen a single working breakout since we started keeping track over a year ago, despite numerous Linux kernel exploits going by in that time, because all the bugs have been in features that we turned off.

https://docs.sandstorm.io/en/latest/developing/security-prac...

That said, there is a cost in compatibility. Most apps can be made to run just fine in this constrained environment, but it does sometimes require tweaks. Docker is more interested in compatibility than in security, so naturally they don't do this kind of attack surface reduction by default. (You can configure it manually, but realistically if it isn't mandatory then few people will bother.)

It is also important to note that Microsoft is not offering shared kernel on Azure using their new container technology because they believe it is too insecure. Microsoft might be a special case because their kernel has an especially large surface area. But I think the current cloud providers that are providing shared kernels are walking a very fine line when it comes to security. Seccomp obviously mitigates this problem and probably puts you in a better position than using a hypervisor if you lock things down very aggressively.

Are linux containers really not securable? That's how Google runs all their stuff in production.

http://research.google.com/pubs/pub43438.html

Google generally doesn't have to worry about mutually-untrusted containers.

The meaning of "Linux containers are not secure" is that untrusted code should not be given root privileges within a container. Google is generally not doing that. They have e.g. trusted Gmail code running on the same machine as trusted YouTube code, handling untrusted emails and untrusted videos. But the Gmail team is not worried about the YouTube team hacking them, or vice versa. The security mechanisms just need to keep honest people honest.

And when they do have untrusted, third-party code to run, for Google Cloud Platform, they use VMs or actual sandboxes: see section 6.1 of the paper you linked.

No, but the Gmail team might be worried about some Russian guy finding a buffer overflow in YouTube's application. Then what? They can escalate privileges and read your email?

Google's deployment of Docker is less susceptible to this. They do additional partitioning of applications.

Yeah... so it seems like Linux containers can be sufficiently hardened?

Not really. You physically seperate gmail and youtube.

Source? How do you get 80% CPU utilization if you have to physically separate all your job types?

http://csl.stanford.edu/~christos/publications/2015.heracles...

Google has way more applications than just Gmail and YouTube.

Basically, if someone breaks into Gmail, you're already in deep trouble. You're not significantly better off because they didn't break into Hangouts. Same in the other direction. So you can run those on the same hardware.

If you think YouTube is less sensitive than Gmail (I don't know if Google actually does), you can separate those, sure. But there are enough applications that are no worse to break into than YouTube, like Photos. You can run those on the same hardware.

At Google's scale it's not too hard to say, this is the stuff we're ridiculously paranoid about and this is everything else, and get 80% CPU utilization on both infrastructures.

Google can afford to keep separate server flights for each app. They have a lot of money and a lot of dedicated computer power for actual revenue generators.

Isn't this specifically a Docker failing? My understanding (from reading rather than use) is that LXC (the original Linux container project) provides a lot more in the way of security than Docker does?

Specially since HP-UX and Tru64 were doing it already in the late 90's.

Absolutely; as I mention in the article. There's nothing new or exciting here; there's simply a big discrepancy between reality and how a lot of users understand it. This is partially true for the my-dev-user-is-part-of-the-docker-group case, but even more so for the container-with-docker-sock-access case.

First let me state that this article is interesting and well written and taught me something new, thank you for that.

However my main frustration is I'm not really sure what the article is advocating for here. Instead of not giving access to the docker daemon to containers (which is legitimately needed for complex deployments where one container needs to dynamically start up "sibling" containers, e.g. a CI service), wouldn't it make more sense to talk about not viewing Docker container security the same as VM security in the first place?

Sure if you're going to do that anyway it makes sense to disable access to the socket, but then there's a million other things you'll have to do because docker containers are currently not primarily intended as a replacement for the isolation security of VMs. Their security is more like a useful extra layer, rather than a full blown replacement.

Check www.hyper.sh and https://github.com/hyperhq/runv

They can boot a new VM with Docker images in 200ms, which is very close to LXC, and perfectly isolated by hypervisor.

The problem of "Virtual Machine" is not "Virtual"/Virtualization, the problem is the full blown guest OS, aka "Machine".

Running docker inside VMs loses all the IO benefits. That's not to say I use docker at all: I don't for this reason. Someday it will be solved and docker will actually be as production ready as its proponents think it is.

Since folks seem to be new to docker and Xen: Hypervisors like Xen have very poor IO performance: one of the major benefits of docket is that it provides containerisation without the IO overhead of a hypervisor. Neither of those have been disputed by anyone, ever.

Reply on news.ycombinator.com