Rootless Containers | Hacker News Reader

Rootless Containers

unit42.paloaltonetworks.com

128 pointspalo29 comments

> Slirp is a well-known project that was originally and mainly used for networking in QEMU (aka Quick Emulator).

Good grief, that's off by almost 15 years. Slirp was originally a SLIP endpoint you could run on a remote dial-in shell account to connect your local system to the internet. As it happens that's basically the same problem needed to get (something like) IP to work out of a userspace context in general, so it got appropriated by the container community and used pervasively. But it's roots go much deeper.

Also... "Quick Emulator"?!

> Also... "Quick Emulator"?!

https://en.wikipedia.org/wiki/QEMU → "QEMU (short for Quick EMUlator)[2] is a free and open-source emulator that performs hardware virtualization."

However, the citation points to an article from The Register circa 2019: https://www.theregister.com/2019/03/11/windows_10_on_kvm/

So who knows?

SLiRP was written by Danny Gasparovski from Canberra.

Actual SLIP needed privs on the host to configure everything. slirp did it in usermode, allowing lowly undergrads to get real IP connections at home from the terminal services universities provided. Then you could run NSCA Mosaic at home :-) And ftp things directly, instead of downloading to your shell account tmp dir and using zmodem to transfer it the last hop.

https://www.linuxjournal.com/article/1174

It went away as real ISPs began to appear, offering PPP connections, and universities dropped their dial-in services. Strange and glorious to see it back again.

It's that exactly what I said?

Just expanding. I was already writing it when your post appeared.

Thank you for pointing that out. You are correct. I'll ask the post author to rectify that sentence. Regarding QEMU, I was also surprised, but I did find some resources using this name. Not sure where it originates though.

I've been following podman for rootless containers. It works fairly well on Fedora. RHEL 8 works, but has older version of packages compared to Fedora.

There are some fairly well documented shortcomings to consider/work-around. You can find that list at https://github.com/containers/libpod/blob/master/rootless.md

The big annoying one in HPC is NFS/GPFS, but that will actually work just fine if you stay as root inside the container, which maps to your uid outside the container.

I'm a little confused by this statement, since at my current employer and previous employer we used NFS mounted home dirs and I would and currently do need to explicitly change my ~/.config/containers/storage.conf to point to a local mount point, such as /var/tmp/containers/$USER in order to perform anything rootless container related. In my case that's with Podman on RHEL 7 and RHEL 8.

That's an acceptable workaround. Just remember the containers aren't available on NFS and thus other systems.

I’m aware, I’m trying to determine what prpl meant by

> but that will actually work just fine if you stay as root inside the container, which maps to your uid outside the container

I'll let Dan Walsh explain it https://www.redhat.com/sysadmin/rootless-podman-nfs

In short, you don't remap when running as root in the container, thus no uid/gid remapping.

I had a derp-out moment yesterday, I totally misinterpreted “as root in the container”. The topic was about rootless containers, so I was completely mentally blacklisting the idea of starting a container from the root user account.

That’s fine until you need to actually write to NFS/GPFS.

Words might not be clicking with me today, I’m trying to specifically decipher this part:

> but that will actually work just fine if you stay as root inside the container, which maps to your uid outside the container

The context of the statement isn’t clicking with me. You start with “it’s a problem with these filesystems” which I’m on board with but then follow with “but it’s fine if you do this” and I’m not seeing the correct correlation.

With user namespaces you can get different uids inside the container. They are usually mapped to something above 65535, in blocks of 65535, according to /etc/subuid. When you run as rootless on podman with uid=1000, you end up at 66535, or 1000 above your start range in /etc/subuid, for example. This is what NFS/GPFS doesn’t like. Locally, in other file systems on the node, it’s fine.

When you run as uid=0 in same the container, you are actually mapped to your uid, since the range starts to be applicable for uid=1. If you can stay as root, you can write to NFS/GPFS just fine.

The issue with this is that lots of available containers have moved away from using things as root. For example, you couldn’t start the vanilla redis container with a data directory on NFS for snapshots because redis runs as user 999.

> When you run as uid=0 in same the container, you are actually mapped to your uid, since the range starts to be applicable for uid=1. If you can stay as root, you can write to NFS/GPFS just fine.

Oh oh oh, you were just saying run the container as root, like literally. That’s what threw me off, because I was in the mindset of rootless. Wasn’t allowing myself to think outside of that environment. Sorry for the confusion on that. Long day yesterday.

Changing uids is something you don't want for HPC use anyhow. Charliecloud is a simple alternative approach specifically for HPC.

Actually, a lot of the time you can get away without namespacing anyhow, just by settings paths to point into an unpacked image. That does depend on what you want to run being sufficiently configurable, though.

Singularity containers go some way towards running without root, though it defaults to setuid setup at boot and dropping privs before execution.

Singularity makes some bad design choices though, like eliminating layers, which results in proliferation of huge container images cluttering everywhere, with no idea what they do. It's really designed for HPC administrators who don't like end users.

https://sylabs.io/guides/3.5/user-guide/security.html

They also have a bad record regarding security, and usability without the setuid scripts is poor.

Well I only see the one bad CVE, but yes security through extra setuid C code seems a bad idea. https://www.openwall.com/lists/oss-security/2019/05/16/1

Ironically, I keep getting told I have to switch over from docker because of security. Plenty of IT security folk have their knickers in a twist at the idea of non-sysadmin priviledged users. They would rather run a giant insecure system (Linux or Windows AD) with lots of LPE and lateral movement than smaller more isolated systems with more trusted users and less LPE. Dev(Sec)Ops concepts still haven't penetrated very deep.

Lack of CVEs for security issues is part of the problem with it. I think SUSE rejected it on the basis of auditing.

While it would be better to run a capability micro-kernel-ish system, you're stuck with Linux for things like HPC. There's no excuse for known exploitable privilege escalations, though and, yes, introducing more opportunity with setuid programs doesn't help. I agree the way such systems are run is unfortunate.

Several DOE sites won't even consider Singularity, including NERSC, who created Shifter (aka Docker + patching - networking) around the same time as development at Singularity started at LBNL (NERSC's parent lab). LANL created CharlieCloud (https://hpc.github.io/charliecloud/) because they needed a hard focus on security first.

There is quite a bit of distrust of Singularity places that value security. I'm hoping podman rootless can eventually come through and render all the bespoke implementations moot, but it will probably be a few years.

> The downside is that V2 doesn’t support all the controllers that were implemented for cgroups V1 (e.g. devices, net_cls, net_prio,etc.).

device controller is present since kernel 4.15.

> Adoption status

Docker/Moby supports FUSE-OverlayFS and cgroup2 as well on master. Planned to be released as v20.0X.

> spearheaded by Podman and LXC.

Yes w.r.t. cgroup, but the network stuffs (slirp4netns & RootlessKit) were originally written for Docker/Moby :)

> the truth is that container engines run Slirp without the seccomp support

Untrue. https://github.com/containers/libpod/blob/d4a3c05c0fcd0c53fa... https://github.com/moby/moby/blob/b47e74255811b2ead92b222541...

We are using Prisma Cloud at work. I'm always surprised to see such a half user friendly tool.

I thought, when i used that tool for the first time, that is some small company building it but nope its paloaltonetworks.

Prisma cloud Was twistlock before it was bought by Paloalto, a small 120 people company.

Actually only the Prisma Cloud Compute part is Twistlock

https://github.com/nanovms is another unikernel attempt to solve the security problem, is an actual VM runnable on type1 hypervisor.

In contrast, gvisor, needs ptrace or kvm.

Both nanovms and gvisor aim to provide an abstraction through emulated system calls.

How does it relate, security wise, to mirageos? I know it's apples and oranges, but only on security dimention scale, is one far better than the other or hard to say?

Reply on news.ycombinator.com