HNHacker News
TopNewBestAskShowJobs

jpetazzo

724 karma · joined July 26, 2010

submissionscomments
jpetazzo··on Docker can now run within Docker
Yeah... Note for future self: make all the mandatory lame jokes in the blog post itself, so people don't feel obligated to do them on HN instead. ;-)
jpetazzo··on Docker can now run within Docker
Are you looking forward to run the Docker CLI in Windows (i.e., being able to use Docker commands from a Windows machine, while the containers run on a remote box), or run the containers themselves on a Windows box?
jpetazzo··on Docker can now run within Docker
Depends what you have in mind, I guess.

You could run most essential system components (syslog, ssh, cron...) inside containers, and bind-mount /home when it makes sense. Since those components need elevated privileges, I don't know if containers would be that useful... And then systemd already puts services in cgroups; it's a nice approach (IMHO).

Or you could use containers a bit like virtualenvs. If you're a Python developer familiar with virtualenvwrapper, you probably use mkvirtualenv when you start working on a new project (or on a different setup for an existing project). Containers could be used in a similar manner with some clever wrappers.

jpetazzo··on Docker can now run within Docker
We're on the same page. That's my next hack — an OpenVPN server within Docker. The blog post, Dockerfile, and helper scripts should be out next week :-)
jpetazzo··on Docker can now run within Docker
Yes, you can do that; I would personally ship the containers separately and provide minimal orchestration. Why? Because if you bundle multiple containers together, they will run on a single host. And if they run on a single host, it's generally easy enough to craft your own ad hoc orchestration.

But if you have multiple containers that require really weird, specific hoops to run together, it's definitely an option!

jpetazzo··on Docker can now run within Docker
Cgroups and namespaces. Even though some namespaces are nested (typically, the PID namespace) why others don't need to be.

There is nothing special with nesting cgroups and namespaces from the kernel point of view. The interesting part is to work around the little details (as anybody who ran nested LXC containers will know) :-)

jpetazzo··on Docker can now run within Docker
This is one of the many potential applications of the new "-privileged" flag of Docker. This is very exciting, because it means that we will soon run Docker CI within Docker itself (instead of ad-hoc VMs), among other things!
jpetazzo··on Docker: Git for deployment
If you are on Linux 3.8, lxc-attach should work.

I use it on a regular basis and never saw any problem (as long as I was on Linux 3.8).

The exact syntax is "lxc-attach -n $FULL_CONTAINER_ID", and you can get the full container ID with "docker inspect" (or "docker ps -notrunc").

Agreed on the my.cnf points, indeed :-)

jpetazzo··on Containers and Docker: how secure are they?
I wouldn't consider it "widely deployed" compared to the Linux installed base.

Sure, Joyent (and others) has "deployed hundreds of thousands of zones into production over the years", but you guys are the only well-known, large-scale, public hosting service using zones (and you're damn good at that, no doubt about it!)

Now compare with dotCloud, Heroku, Dreamhost, 1&1, Mediatemple, OVH, Amen (just to name those I can remember without doing an extensive research): those guys have also "deployed hundreds of thousands of Linux-based VPS into production", using VServer, OpenVZ, and more recently, LXC.

Don't get me wrong: I'm a big fan of Solaris (and its heritage); I have lots of my marbles on ZFS; I hacked basic ZFS support in Docker just for fun a while ago; and if I knew better, I would love to find a way to run sub-zones and a ZFS pool in a Joyent SmartOS instance and port Docker to your platform. But there is a helluva lot of Linux hosters out there.

I'll close with a lovely paraphrase: « Perhaps LXC containers are a new technology for you, but understand that it's not new for everyone; some of us have been doing this for a while — widely deployed and in production. » ☺

jpetazzo··on Containers and Docker: how secure are they?
Regarding the lxc.drop vs lxc.keep: of course, we eventually want to switch to the latter, since it's obviously better to "deny all, then allow some" than the opposite. And I can only be grateful that you provided those elements. It looks like you think like your contribution wasn't taken into account, but it definitely was, and I'm sorry that you feel that way.

So why does Docker still ship with lxc.drop? Well, a large number of people are still using LXC 0.7, which doesn't support lxc.keep, AFAIK. But it is very likely that Docker 1.0 will either require LXC 0.9, or totally get rid of LXC userland tools, or provide multiple implementations depending on what you have installed locally; and then lxc.keep will definitely kick in.

Also, the initial security choices of Docker represent a middle ground between "lock down everything" and "allow anything to happen". It had to be secure enough so that people could run regular app servers with a reasonable level of trust; and permissive enough to allow e.g. normal package managers to run.

Moreover, Docker is evolving: we recently added the "-privileged" flag (available in the master branch, and very probably in 0.6.0, due in a few days), allowing to switch between a more secure configuration, suitable for e.g. public PAAS environments, and a more permissive configuration, suitable for private PAAS, continuous integration, that kind of things. And this is just one step in that direction.

jpetazzo··on Containers and Docker: how secure are they?
I disagree with this assertion that "VMs will always be more secure". Of course, they bring an extra layer (or rather, a layer of different nature).

But check the number of Xen vulnerabilities (I kept up with those for a while because I still run a Xen cluster): they are very real. And keep in mind that Xen (at least in my case!) doesn't bring an extra layer of security: if you are (e.g.) an IAAS provider using Xen to sell VMs, your customers can run anything they like in their VMs, and Xen will be the only layer. Your hypervisor will be "on the front line" if you see what I mean.

I would actually argue quite the contrary. I.E.: exploits affecting containers are likely to be exploits affecting all Linux systems, meaning that they will draw much more attention and scrutiny than exploits affecting hypervisors, and they are likely to be fixed faster.

jpetazzo··on Containers and Docker: how secure are they?
True, I will update the blog post so that it feels less misleading. Source code for Solaris zones is indeed available; but I wouldn't consider it widely deployed. Of course, some people are using it in public hosting environments (the most notable example is probably Joyent); but I don't think that it's significant compared to the installed base of VServer, OpenVZ, or LXC out there.

I mean — it's trivially easy to get access to a Linux VPS, for a ridiculously low price (sometimes for free). Now compare with something equivalent based on Solaris zones.

But yeah, I'll definitely update the blog post, thanks!

jpetazzo··on Containers and Docker: how secure are they?
Are you implying that gaining root access inside a LXC container means that you can escalate to the host system, or to sibling containers?

If yes, I would like to see an example of that (that works on systems with very minimal lockdown, i.e. using the device control group and kernel capabilities).

Otherwise, if you just mean that "0-day Linux root vulnerabilities can be used to escalate from non-root to root in a Linux Containers", that's a truism, and it also stands true for VMs or OpenVZ systems.

Just like 0-day vulnerabilities will help people to escalate from non-root to root in a FreeBSD jail or Solaris zone.

jpetazzo··on Containers and Docker: how secure are they?
By the way, if anyone knows of a documented exploit for LXC, I would love to hear about it. People (generally advocating VMs, zones, jails, OpenVZ...) will often say that "containers are not secure", but once you've taken some basic steps (like locking down kernel caps and device access) it becomes difficult to find an actual threat.
jpetazzo··on Docker interactive tutorial
It's basically what you get when you start your deployment toolchain as simple shell scripts, then make them evolve, add features, and at some point, rewrite from scratch with a "real" programming language. I don't know if that matches your definition of "glorified" though :-)
jpetazzo··on Docker interactive tutorial
Actually, I've ran Docker within QEMU within Docker (using v9fs so that QEMU could use the container's FS as root FS). Works, but painfully slow and not very resource-efficient :-)
jpetazzo··on A first look at docker.io
Sorry, but this is FUD:

> IF you have root on the container, you have root on the host.

This is only true on badly configured systems. If you run some kind of public shared hosting (like Heroku, dotCloud, etc.) you probably slap some extra security on top of it. For instance, dotCloud uses GRSEC, limits root access, and uses kernel capabilities.

It won't take years to get LXC in production for shared hosting: it has been in production for shared hosting for years -- but by people who (more or less) knew what they were doing.

Agreed, "out-of-the-box LXC" is probably not that secure; which is probably why many people won't deploy it. And I can't blame them. Any technology generally starts being usable (or usable safely) only for expert users, then progressively gets more industrialized and ready to use for a broader audience. It doesn't mean that the technology is not mature.

Also, the user separation that you mention has been implemented in the Linux kernel for a while[1]; it's called "user namespace", and even if the default LXC userland tools do not make use of it at this point, it's here.

[1] https://wiki.ubuntu.com/UserNamespace

jpetazzo··on A first look at docker.io
I don't know if they meant GB or Gb, but for the record, I didn't see any difference in disk performance between native and containerized apps. In that case, that was pulling ~900 MB/s seq reads from a RAID10 disk array of 8 old 7k HDD. This is not surprising, as the code path for block I/O is exactly the same for native and containerized processes.
jpetazzo··on Deis: Open Source PaaS based on Docker, Chef, and Heroku Buildpacks
I'm not involved in Flynn nor Deis, but since Flynn is not available yet (and no source is visible at this point), I would be surprised if anyone could tell :-)
jpetazzo··on Node.js Deployments with Docker, Dokku, and Digital Ocean
You should be able to run with just 512 MB, as long as your app can do with that. Docker itself as a tiny footprint, and containers have almost zero overhead :-)
jpetazzo··on Docker Desktop: Your Desktop over ssh inside of a Docker container
Root on the container doesn't mean root on the host. Machine-level virtualization has received more scrutiny than LXC, so as of today, many people consider traditional VMs to be more secure. But KVM or Xen are not intrinsically more secure than LXC or OpenVZ. They all have their histories of exploits and privilege escalations.

One key thing is, that it makes sense to run containers without root privileges (greatly improving security), while it is much harder to realistically run a VM without root processes. As a result, it could be said that containers are much safer, because before even thinking about breaking out of the container, you have to work on a root exploit - on a system which, by essence, only runs the processes that you really need, and has a much smaller attack surface.

We're working on a more elaborated answer, to be included within Docker docs!

jpetazzo··on Docker Desktop: Your Desktop over ssh inside of a Docker container
Actually, this: https://launchpad.net/~dotcloud/+archive/lxc-docker
jpetazzo··on Dockerlite: Lightweight Linux virtualization with BTRFS and LXC
Actually, Docker just got the "bind mounting" feature that you are requesting. If you want the long and boring details, it's in this pull request:

https://github.com/dotcloud/docker/pull/602

Dockerlite doesn't have those features, though (keep in mind that dockerlite is just a playground for funky features, nothing else!)

jpetazzo··on Dockerlite: Lightweight Linux virtualization with BTRFS and LXC
If you want to know more about the JSON parser, it comes from https://github.com/dominictarr/JSON.sh.
jpetazzo··on Dockerlite: Lightweight Linux virtualization with BTRFS and LXC
I take full responsibility for all the horrendous orthographic, grammatical, or otherwise language-related mistakes committed in this README :-)
jpetazzo··on Dockerlite: Lightweight Linux virtualization with BTRFS and LXC
The JSON parser actually comes from https://github.com/dominictarr/JSON.sh (as mentioned in dockerlite's README file).

The JSON parser is not "the main part of this project"; it was included just in case we needed to interact with the registry (but it wasn't deemed necessary).

jpetazzo··on Dockerlite: Lightweight Linux virtualization with BTRFS and LXC
Yes, dockerlite is mainly a proof-of-concept, to experiment with a couple of important things.

1. Use BTRFS instead of AUFS, and see if any specific problem arises, or if we hit any corner case when doing that.

2. Setup the network without using LXC default userland tools, in a race-condition free way. This is not as obvious as it sounds.

dockerlite was a success on both accounts. It paved the way for BTRFS support, and gave us some insights about how to make the network setup more flexible.

jpetazzo··on Docker joins the Linux Foundation
The main blocker is AUFS, but Docker has been shown to run fine with overlayfs and BTRFS; and the latter is available on virtually all recent Linux systems (it's even usable on RHEL kernels, which aren't really known for being bleeding-edge).

The hardest part is to provide interchangeable, pluggable storage systems.

jpetazzo··on Docker joins the Linux Foundation
raises a hand

In the early 2000s, I was in the hosting business, and we started using UML (user-mode-linux), then vserver and openvz. When Xen came around, we switched to that, because it made more sense for us (and for our customers): more control, more "this is your machine" feeling.

"Yes, but containers are better because they are lightweight!" Of course, but this idea was pointless back then, because people who wanted really lightweight things went to shared hosting (and for massive PHP hosting, containers can't beat a single Apache instance serving tens of thousands of vhosts; at least not yet).

Maybe (maybe!) things changed with the rise of stacks like Python/Ruby/Node.js/etc for which there are no cheap shared hosting solutions like there is for PHP. Then containers are obviously cheaper than running your own VPS, and easier to setup (and isolate) than shared hosting on a single box.

jpetazzo··on Build Your Own SaaS Using Docker (cs50x project)
I'm not sure what you're referring to when you say "such young OS". Solaris zones and ZFS have been around for almost a decade. Docker is young, but it wraps around LXC, cgroups, AUFS, and namespaces. Cgroups and AUFS were introduced in 2006. Namespaces are more recent, but they are basically a reimplementation of concepts from OpenVZ and VServer (and the latter is more than 10 years old). FreeBSD jails are great, and they have also been around for a while; but they are still lacking a lot of features (a jail doesn't have its own network stack, and resource accounting and limiting isn't as evolved as what cgroups provide), and unionfs is far from being as stable as AUFS.

Don't misunderstand me: FreeBSD jails are awesome if you're doing FreeBSD and want lightweight virtualization. For the rest of us, LXC is just as awesome, and docker lowers the barrier of entry considerably!

My 2c.

← PreviousPage 2 of 3Next →