I hope that's not it. Talk about throwing out the baby with the bathwater.
It's true that Rust's standard library panics on ENOMEM, but not only is this not inherent to the language, it's possible to extend the standard library, and folks are doing so. See https://github.com/rust-lang/rust/issues/48043 for support of "fallible allocations".
Three years later, I'm comfortable with this decision; three more years of waiting on Rust 1.0 would have killed it as a language. Many, many useful programs have been written in Rust since then.
Of course, you can disagree with this decision; I mostly want to communicate that it was not a flippant "who needs to care about that," but a careful decision made after weighing the tradeoffs. In the old days, those tradeoffs were very different.
As a bonus, this means that when you're reading code you can easily spot the function calls that have a possibility of failing.
> Zig's standard library functions that require allocation take the allocator as a parameter, and return oom errors back up to the caller to do with as they please.
You could do something like this in Rust. Here's a function that accepts a vector and pushes something onto it:
fn foo(v: &mut Vec<i32>, item: i32) {
v.push_back(item);
}
Does the data structure hold the allocator, or the function? What if I created a list with one allocator, but called push_back with another? It'd have to be in the structure, right? Given that, here's what this feature would look like, roughly, if Vec<T> supported it:
fn foo<A: Allocator>(v: &mut Vec<i32, A>, item: i32) -> Result<(), Box<Error>> {
v.push_back(item)?;
}
// before
foo(&mut v, 5);
// after
foo(&mut v, 5)?;
pub fn ArrayList(comptime T: type) type {
return struct {
const Self = @This();
raw_items: []T,
len: usize,
allocator: *Allocator,
// ...
pub fn append(self: *Self, item: T) !void {
const new_item_ptr = try self.addOne();
new_item_ptr.* = item;
}
pub fn addOne(self: *Self) !*T {
const new_length = self.len + 1;
try self.ensureCapacity(new_length);
return self.addOneAssumeCapacity();
}
// ...
};
}
test "std.ArrayList.basic" {
var bytes: [1024]u8 = undefined;
const allocator = &std.heap.FixedBufferAllocator.init(bytes[0..]).allocator;
var list = ArrayList(i32).init(allocator);
try list.append(1);
try list.append(2);
try list.append(3);
assert(list.len == 3);
}
[1]: https://github.com/ziglang/zig/blob/e3d8cae35a5d194386eacd9a...
[2]: https://ziglang.org/documentation/master/#Error-Set-Type
And yeah, if you hold a pointer to the allocator instead of parameterizing by it, that helps too. With that in mind,
fn append(&mut self, item: i32) -> Result<(), Box<Error>> {
v.push_back(item)?;
}
Anyway, yeah, if we allowed for inferring the error type, that would make this almost the same as Zig here. Stealing your syntax:
fn append(&mut self, item: i32) -> !() {
v.push_back(item)?;
}
https://github.com/ziglang/zig/blob/master/std/debug/failing...
Ever since Unix had copy-on-write data pages, it has had effectively the same behavior around OOM. The semantics of fork() require it, unless you're OK with the idea of a process that uses more than half of the available memory being unable to launch any child processes. So the idea of killing processes on OOM is very old indeed.
Anyway, whether overcommit is a good idea is somewhat tangential. The point is that portable programs cannot rely on being able to detect and recover from OOM in the same process. That's why you see so few programs try. (The programs that do try frequently suffer from security vulnerabilities in their OOM handling paths, ironically enough.)
vfork is better than fork/exec with overcommit.
It's interesting to me that you mention portability as an argument for why an out of memory situation should cause a crash rather than be handled like any other error.
Let's say I make a library and I want this library to be maximally portable. That means it should run in some of the following environments:
* Microsoft Windows
* embedded software
* operating system kernels
* drivers
* real-time software
In all of these environments, a library that handles out of memory correctly would be more portable than a library that panics.
If only there was a programming language that helped programmers solve this problem, while preventing the pitfalls that result in these security vulnerabilities that you mention...
I rarely write Windows-only programs, so I can't rely on OOM handling in my logic. It's much simpler to just have the same OOM story everywhere (crash).
//kick any error up the call stack
var buffer = try allocator.alloc(u8, 500);
//in debug builds, assert that this will not fail
//in release-fast failure will result in undefined
//behavior.
var buffer = allocator.alloc(u8, 500) catch unreachable;Besides the standard library, it also has the core library, which is a small subset that is appropriate for all environments, even those without allocation or those with different strategies for allocation.
With this goal in mind, many of the decisions make sense. A portable application can't rely on getting errors on out of memory situations. Many people don't write code that handles out of memory, or if they do they don't test it, which can lead to significant bugs. Error handling for out of memory is cumbersome if it is required for every operation which could allocate memory, such as pushing to a Vec.
Additionally, Rust 1.0 didn't have some of the later features to make error propagation simpler, so functions which could return errors were more cumbersome to use.
And finally, fallible allocation can be added after the fact. It is being worked on: https://github.com/rust-lang/rust/issues/48043
For now, anyone who wants to use Rust in those situations, but still wants allocation support, needs to implement their own allocator and collections with fallible operations. Once the allocator API and fallible collections support lands, much more of the Rust standard library will be usable in such an environment.
The system administrator can turn it off, kind of. But a zig or rust program running in user space without privileges cannot turn it off, not even for the program itself.
Do you run on Linux? Do you leave vm.overcommit_memory at the default 0? Do you run on 64-bit platforms? Most people here I imagine would say yes, yes, and yes. If so, ENOMEM is not a thing that happens. Some process on the machine will get OOM-killed instead. It's silly to write a bunch of dead code.
Ignoring my other point (that ENOMEM doesn't happen on the most common deployment environment), what do you think the correct ENOMEM behavior should be?
In many situations, I think it's to crash (what Rust does anyway). For example, at work I write cloud servers, which are replicated. If there's say a memory leak that is causing it to have not quite enough RAM, I'd rather it go down and restart cleanly that try to limp on. The other servers will take the load in the meantime. (They might all crash in waves, which is sad but better that than all being unable to serve well and lingering in that state. If you don't crash, then you need some external agent that decides when to restart them, and you end up with a pretty similar outcome with more complexity and rarely exercised logic.)
There are certainly specialized environments where you want to do something else, but it sounds like you're saying it's thoughtless/wrong for the program to crash, and in general I disagree.