Comment by pornel | Hacker News Reader

Comment by pornel | Hacker News Reader

Parent Full thread

The "but the OOM killer!" argument has been a disaster for Rust, and keeps derailing all design discussions.

• Linux is not the only OS in the world.

• Even on Linux you have containers/cgroups that can impose hard limits.

• Platforms without virtual memory are also an important target for Rust.

• On 32-bit platforms you can run out of address space before you run out of RAM.

• Regardless of what the OS does, the application may still want to impose its own internal limit (e.g. https://lib.rs/cap) to avoid being OOM-killed or swap death.

People keep telling me how Linux never runs out of memory, while I'm currently firefighting a torrent of coredumps caused by Rust's self-own on OOM that actually happens.

I generally love Rust, but its OOM handling is awful, and the "but the OOM killer" nonsense is to blame for stalling the absolutely critical fixes it urgently needs.

> but the OOM killer!" [..] disaster for Rust [...]

Only on HN discussions (the disaster part) ;=)

(Through it's is a broken discussion, OOM on Linux is broken, but it becomes increasingly "un-"broken and like you said other platform exists, too).

Anyway there is no disaster as:

Rust does support handling memory allocations gracefully, the allocator API defaults to this!!

Just various types defined in [lib]std (or precise [lib]alloc) default in there default methods to calling the (replaceable!) allocation error hook.

This is due to ergonomics, there is no sane/ergonomic way for "common non embedded/kernel programming or non special purpose use-cases" to make literally every method which potentially allocates return a result. And outside of such special purpose cases you can always recover from panics (i.e. outside of special purpose cases panic=abort is a anti-pattern).

Still, even in std, methods like `try_reserve` exists and are used, e.g. by serde deserializers to make sure to gracefully fail if the user tries to allocate a 10GiB array (because compressed data formats ;-) ).

BUT for the kernel all this doesn't matter as it is very unlikely it will use [lib]std or [lib]alloc. So they can default to just always implement their types in a way which doesn't panic on memory allocation.

And wrt. to panic in other (basically guaranteed to be a bug cases) panic=BUG() (ther kernel BUG() macro) can be done.

I'm more worried about the float/128bit integer thing he mentioned, as I don't know anything about it. I assume it's not just a miss understanding of overflow checks and other debug assertions the kernel likely will disable for non-debug builds?

> I'm more worried about the float/128bit integer thing he mentioned, as I don't know anything about it. I assume it's not just a miss understanding of overflow checks and other debug assertions the kernel likely will disable for non-debug builds?

Idk about 128bits integer, but the floating point question has been discussed on Rust's subreddit[1]:

> Normally, the kernel leaves the floating-point state of the CPU in whatever state the userspace process left it in, so that you can do a quick system call without having to save and restore all that state. It's possible to use floating point in the kernel with a great deal of care, but you have to notify the kernel that you're doing so, so that it can save the userspace floating-point state.

> So, it'd be helpful if by default Rust didn't allow use of floating-point, and then you could opt in to allowing it for specific code.

And from Linus[2]

> In other words: it's still very much a special case, and if the question was "can I just use FP in the kernel" then the answer is still a resounding NO, since other architectures may not support it AT ALL.

[1]: https://www.reddit.com/r/rust/comments/mqxr1a/rfc_rust_suppo...

[2]: (from the same reddit thread) https://ipfs.io/ipfs/QmdA5WkDNALetBn4iFeSepHjdLGJdxPBwZyY47i...

Well if that it all this could be solved by rust by having a "floating_point_usage" (or similar) lint which normally defaults to "allow" but for kernel devs defaults to "error" (which could be done through an option in the target spec, which as a side note is also what defines if you have hard,soft or no fp support). (there are probably better solution but this one should be the easiest).

There's actually a lint specifically intended for that already. `float_arithmetic`:

https://rust-lang.github.io/rust-clippy/master/#float_arithm...

# What it does

Checks for float arithmetic.

# Why is this bad

For some embedded systems or kernel development, it can be useful to rule out floating-point numbers.

Thanks, nice, but for the kernel use case it would need to be part of rustc.

Simply install clippy and call "clippy-driver" instead of "rustc", which will handle this. Clippy is largely the lints that haven't either made it into rustc or are fairly edge-case-y.

Why? The Rust-for-Linux team already made Clippy a requirement for Rust kernel code. For niche configuration that are really project-specific like this, it makes total sense to use external tooling to ensure the guidelines are respected, it doesn't need to be the compiler enforcing these guidelines.

The overcommit is not the only argument.

The second argument is that complex programs that need to handle OOM properly will be full of complex untested error paths.

How should one handle a OOM error If one can't allocate some temporary node during an algorithm and how to recover without leaving a corrupted state? That's not a question I want to answer for almost every function call.

That's why the rust standard library data structures don't handle OOM. To make it easier to use for the most common case. And for the cases where the OOM handling is required, one can simply use data structures for it, which is what Linux will do (problem solved)

Also compare that to the case of mutex poisoning which can return an error on each unlock, and which is now considered a mistake because in practice, nobody want to handle a poisoned mutex error.

I argue it's a rare-enough problem:

• Statistically, likelihood of failure is proportional to the allocation size. You'll most often fail to allocate a very large Vec, and then only need to allocate a few bytes for a string that says "error!". I'm using https://lib.rs/fallible_collections and just a few strategically placed try_reserve has brought down my coredumps by 99%.

• Panic on OOM would unwind, and unwinding tends to free memory rather than allocate anything. The standard library should reserve/preallocate enough memory to be able to start unwinding.

• Rust is good at avoiding temporary allocations and relies on the stack a lot. You don't have to use fancy error-handling libraries that collect backtrace on every error. Error handling with enums doesn't touch the heap at all.

• Even if everything fails, and you get OOM during OOM handling, that's fair, and still better than abort every time.

Note that Rust currently doesn't handle OOM by panicking. It unconditionally aborts the whole process. Libstd could switch to panics without API changes, and that would make OOM handling possible via catch_unwind.

> Panic on OOM would unwind,

OR abort the process.

Panics are for fatal irrecoverable errors, not for recoverable errors.

For example of the problem that the API would have if it wanted to handle OOM: Box, String or Vec, the most basic types in the rust alloc crate, couldn't implement Clone, because clone() can't fail, and you need to allocate memory to clone these data structures. As a result, lots of generic algorithms that relies on on their type to be clonable wouldn't work, and most user types could not #[derive(Clone)] anymore.

Hows that' a problem?

Provide `TryClone`, there, problem solved.

I agree, it doesn't seem like these types can actually implement Clone if allocation can fail.

So, if I get this right, this means that the "just don't use the alloc crate" argument given somewhere else here is not practical ?

No not at all.

Because ...? I mean, from the perspective of an outsider reading that you "can't use strings" if there's a risk of allocation failure is a bit worrying.

Because you just don’t use the types in the alloc crate, or you use the (only partially implemented) APIs that return a Result<> instead of panicking on allocation failure.

For that kind of low-level work I’d probably do the former and implement my own data structures, which is what you do with no_std currently.

As others have said there’s nothing in Rust the language which assumes allocations always succeed. The standard library made the choice to panic on allocation failure, which was the right choice in order to make an ergonomic API that’s suitable for higher-level work. If you’re not using the standard library (which currently includes people on eg embedded targets) then you can handle allocation any way you like. And if you’re writing drivers for the kernel you’d assume you’re ok handling that.

GP was pointing out that the Clone trait doesn’t allow for returning a Result, so you can’t use that trait to clone anything that allocates. But that’s not really a big deal since you can just make your own TryClone that does.

Ok, thanks for the clarification.

No. People that care about catching OOM failures will simply have to use other crates that have a more complex API, but that's what they want

I was just saying it is a right choice in the alloc crate, which is meant for user space programs, to simply panic on case of OOM, because otherwise the API would be more complex for everyone.

I agree that the OOM killer is frankly irrelevant in this discussion (and a bad idea in the first place IMO, cue the "airplane company deciding which passenger to throw out the plane" argument).

But on the other hand in my experience with C and C++, languages which do allow explicit OOM handling, most applications either just crash when that happen (the `xmalloc` route) or attempt to recover but often really can't do much.

It goes even beyond programming languages. Very few environments deal nicely with near-OOM conditions. My Linux desktop becomes effectively unusable if I start aggressively swapping. And technically at this point as long as I have swap I'm not literally out of memory.

My general point is not that I don't think Rust would benefit from allowing the developer to explicitly handle OOM conditions, it's more that the vast majority of applications are not written to gracefully degrade in these conditions anyway, and if the only handling you do is effectively `alert("out of memory!"); exit(1);` it's really not worth bothering with it.

On the other hand for the minority of applications that want to do something more meaningful on OOM you probably want to take the time to design some ergonomic system that won't litter the code with boilerplate, i.e. you probably don't want every single Vec and String and basically every single facet of the standard API that can allocate behind the scenes to return a `Result<_>`.

I'm sure there's something to do, but I think the Rust devs are right not to rush it given that there are so many other features left to implement.

> if the only handling you do is effectively `alert("out of memory!"); exit(1);` it's really not worth bothering with it.

I'd argue that even this is worth it, for most software. At least then you get a clean exit and an obvious problem. The alternative is to not detect it and let the code walk off into undefined behaviors and potentially subtle bugs that can harm data and/or break security.

Yeah, explicit OOM exit beats random NULL deref any day. The former can be analyzed by an L2 tech out of a logfile. The latter requires spinning up GDB to figure out where NULL came from.

In my experience the problem with the latter isn't even that it requires a skilled analyst & a debugger. The reality is that, in a multi-threaded program, by the time your process has crashed on a null dereference the offending call stack is probably long gone.

The faster you fail the more readily apparent the root cause will be from your logs/core dump/etc. For this to work though reboots (of the application/environment) need to be inexpensive, and you also need supervision. That supervision can either be at the OS level, like Linux's systemd, Solaris' SMF, Apple's launchd, etc., or at the application level like Erlang/OTP.

Yes, that's another great reason to fail promptly.

To be clear Rust will give you a clean exit in this situation (or at least, I assume that to be true, it would be a glaring issue if it didn't). On linux however since malloc usually can't fail you won't ever see that since the kernel will realize the issue on access and not on alloc, then the OOM killer will play russian roulette with your processes.

It would be really helpful, if the OOM killer should be adjustable to kill non-overcommitting programs last (kill any with overcommit first). Then programs could be slowly fixed/improved.

word

Reply on news.ycombinator.com