Hacker News Reader

Comment by chubot | Hacker News Reader

So I guess she is talking about this case:

    1) something like systemd starts a process in a cgroup, to limit memory usage
    2) it disables the kernel OOM killer, and expects to
       receive notification itself and kills the offending process
    3) systemd crashes?  This it can no longer process the
       notifications, so you have a child process which is at
       the memory limit but not killed.

This is a great example of why an init system should be as simple as possible and not die under any circumstance :) Why would your init system die? If you something like daemontools, they are designed to not even allocate memory after startup, because that could fail.

Was she talking about systemd? Which process in systemd does the killing? Is it systemd-nspawn or something? (I'm not using systemd now.)

View on HN

Is systemd going to become like the NSA leak? Every article no matter what will somehow become about it?

This has nothing to do with systemd.

And init of systemd doesn't die anyway, all the extra stuff is run in separate processes, process 1 is indeed, very simple.

systemd is the most notable process manager that has cgroups support, and the article is talking about process managers that "fail".

It doesn't matter if it's not the PID 1 that fails. The article is saying: if the process that is supposed to receive OOM notification and kill the process fails, you will observe symptoms that are very hard to diagnose.

PID 1 dying is of course really bad, but if other processes in the init system die, your system can still be hosed, as pointed out here.

> systemd is the most notable process manager that has cgroups support

So? Not everything is about systemd.

systemd does not disable the OOM killer, so this has nothing to do with systemd.

You can create cgroups yourself you know - you don't need systemd for that.

> So? Not everything is about systemd.

It's time for an equivalent to Godwin's law for systemd.

I don't think the standard Linux kernel supports user-space OOM handling. That's a patch set that Google has been pushing for a few years, but that hasn't been accepted yet (as far as I know). So I'm guessing she's talking about Google's server infrastructure.

My understanding is that the program running in the cgroup is disabling the OOM killer and doing it itself, so only whatever is inside of the cgroup is frozen (Presumably not systemd). Most programs have no need to disable the OOM killer anyway so it's unlikely they'd run into this problem anyway (They'd just get killed off instead of hanging things).

I don't think that's how that works. A process inside a cgroup has consumed all of the cgroup's RAM. Outside that cgroup you run `ps aux`, which will list all processes, including those inside the container, and this will hang forever. Unless, I am misunderstanding it, and each cgroup gets its own /proc filesystem.

This is only an issue when you disable OOM killer. If your OOM killer replacement dies, then you have a problem, and you shouldn't have disabled it in the first place.

The kernel isn't going to protect you from turning off critical bits.

Yes, I read the article. My point is that a container that ran out of RAM and is running a buggy process manager affects the entire system, not just the container.

Reply on news.ycombinator.com