724 karma · joined July 26, 2010
You could run most essential system components (syslog, ssh, cron...) inside containers, and bind-mount /home when it makes sense. Since those components need elevated privileges, I don't know if containers would be that useful... And then systemd already puts services in cgroups; it's a nice approach (IMHO).
Or you could use containers a bit like virtualenvs. If you're a Python developer familiar with virtualenvwrapper, you probably use mkvirtualenv when you start working on a new project (or on a different setup for an existing project). Containers could be used in a similar manner with some clever wrappers.
But if you have multiple containers that require really weird, specific hoops to run together, it's definitely an option!
There is nothing special with nesting cgroups and namespaces from the kernel point of view. The interesting part is to work around the little details (as anybody who ran nested LXC containers will know) :-)
I use it on a regular basis and never saw any problem (as long as I was on Linux 3.8).
The exact syntax is "lxc-attach -n $FULL_CONTAINER_ID", and you can get the full container ID with "docker inspect" (or "docker ps -notrunc").
Agreed on the my.cnf points, indeed :-)
Sure, Joyent (and others) has "deployed hundreds of thousands of zones into production over the years", but you guys are the only well-known, large-scale, public hosting service using zones (and you're damn good at that, no doubt about it!)
Now compare with dotCloud, Heroku, Dreamhost, 1&1, Mediatemple, OVH, Amen (just to name those I can remember without doing an extensive research): those guys have also "deployed hundreds of thousands of Linux-based VPS into production", using VServer, OpenVZ, and more recently, LXC.
Don't get me wrong: I'm a big fan of Solaris (and its heritage); I have lots of my marbles on ZFS; I hacked basic ZFS support in Docker just for fun a while ago; and if I knew better, I would love to find a way to run sub-zones and a ZFS pool in a Joyent SmartOS instance and port Docker to your platform. But there is a helluva lot of Linux hosters out there.
I'll close with a lovely paraphrase: « Perhaps LXC containers are a new technology for you, but understand that it's not new for everyone; some of us have been doing this for a while â widely deployed and in production. » ☺
So why does Docker still ship with lxc.drop? Well, a large number of people are still using LXC 0.7, which doesn't support lxc.keep, AFAIK. But it is very likely that Docker 1.0 will either require LXC 0.9, or totally get rid of LXC userland tools, or provide multiple implementations depending on what you have installed locally; and then lxc.keep will definitely kick in.
Also, the initial security choices of Docker represent a middle ground between "lock down everything" and "allow anything to happen". It had to be secure enough so that people could run regular app servers with a reasonable level of trust; and permissive enough to allow e.g. normal package managers to run.
Moreover, Docker is evolving: we recently added the "-privileged" flag (available in the master branch, and very probably in 0.6.0, due in a few days), allowing to switch between a more secure configuration, suitable for e.g. public PAAS environments, and a more permissive configuration, suitable for private PAAS, continuous integration, that kind of things. And this is just one step in that direction.
But check the number of Xen vulnerabilities (I kept up with those for a while because I still run a Xen cluster): they are very real. And keep in mind that Xen (at least in my case!) doesn't bring an extra layer of security: if you are (e.g.) an IAAS provider using Xen to sell VMs, your customers can run anything they like in their VMs, and Xen will be the only layer. Your hypervisor will be "on the front line" if you see what I mean.
I would actually argue quite the contrary. I.E.: exploits affecting containers are likely to be exploits affecting all Linux systems, meaning that they will draw much more attention and scrutiny than exploits affecting hypervisors, and they are likely to be fixed faster.
I mean — it's trivially easy to get access to a Linux VPS, for a ridiculously low price (sometimes for free). Now compare with something equivalent based on Solaris zones.
But yeah, I'll definitely update the blog post, thanks!
If yes, I would like to see an example of that (that works on systems with very minimal lockdown, i.e. using the device control group and kernel capabilities).
Otherwise, if you just mean that "0-day Linux root vulnerabilities can be used to escalate from non-root to root in a Linux Containers", that's a truism, and it also stands true for VMs or OpenVZ systems.
Just like 0-day vulnerabilities will help people to escalate from non-root to root in a FreeBSD jail or Solaris zone.
> IF you have root on the container, you have root on the host.
This is only true on badly configured systems. If you run some kind of public shared hosting (like Heroku, dotCloud, etc.) you probably slap some extra security on top of it. For instance, dotCloud uses GRSEC, limits root access, and uses kernel capabilities.
It won't take years to get LXC in production for shared hosting: it has been in production for shared hosting for years -- but by people who (more or less) knew what they were doing.
Agreed, "out-of-the-box LXC" is probably not that secure; which is probably why many people won't deploy it. And I can't blame them. Any technology generally starts being usable (or usable safely) only for expert users, then progressively gets more industrialized and ready to use for a broader audience. It doesn't mean that the technology is not mature.
Also, the user separation that you mention has been implemented in the Linux kernel for a while[1]; it's called "user namespace", and even if the default LXC userland tools do not make use of it at this point, it's here.
One key thing is, that it makes sense to run containers without root privileges (greatly improving security), while it is much harder to realistically run a VM without root processes. As a result, it could be said that containers are much safer, because before even thinking about breaking out of the container, you have to work on a root exploit - on a system which, by essence, only runs the processes that you really need, and has a much smaller attack surface.
We're working on a more elaborated answer, to be included within Docker docs!
https://github.com/dotcloud/docker/pull/602
Dockerlite doesn't have those features, though (keep in mind that dockerlite is just a playground for funky features, nothing else!)
The JSON parser is not "the main part of this project"; it was included just in case we needed to interact with the registry (but it wasn't deemed necessary).
1. Use BTRFS instead of AUFS, and see if any specific problem arises, or if we hit any corner case when doing that.
2. Setup the network without using LXC default userland tools, in a race-condition free way. This is not as obvious as it sounds.
dockerlite was a success on both accounts. It paved the way for BTRFS support, and gave us some insights about how to make the network setup more flexible.
The hardest part is to provide interchangeable, pluggable storage systems.
In the early 2000s, I was in the hosting business, and we started using UML (user-mode-linux), then vserver and openvz. When Xen came around, we switched to that, because it made more sense for us (and for our customers): more control, more "this is your machine" feeling.
"Yes, but containers are better because they are lightweight!" Of course, but this idea was pointless back then, because people who wanted really lightweight things went to shared hosting (and for massive PHP hosting, containers can't beat a single Apache instance serving tens of thousands of vhosts; at least not yet).
Maybe (maybe!) things changed with the rise of stacks like Python/Ruby/Node.js/etc for which there are no cheap shared hosting solutions like there is for PHP. Then containers are obviously cheaper than running your own VPS, and easier to setup (and isolate) than shared hosting on a single box.
Don't misunderstand me: FreeBSD jails are awesome if you're doing FreeBSD and want lightweight virtualization. For the rest of us, LXC is just as awesome, and docker lowers the barrier of entry considerably!
My 2c.