Comment by samiv | Hacker News Reader

Comment by samiv | Hacker News Reader

Parent Full thread

There are several levels here.

In your C++ (or C) program you have one (or more) allocators. These are just pieces of code that juggle blocks of memory into smaller chunks for the program to use. Typically the allocators get their memory from the OS in pages using some OS system call such as sbrk or mmap.

For the sake of argument, let's say I write an allocator that has a limit of 2MiB, while my system has 64Gib or RAM. The allocator can then fail some request when it's internal 2MiB has been exhausted. In C world it'd return a nullptr. In C++ world it would normally throw bad_alloc.

If this happens does this mean the process is out of memory? Or the system is out of memory? No, it doesn't.

That being said where things get murky is because there are allocators that in the absence of limits will just map more and more pages from the OS. The OS can "overcommit" which is to say it gives out more pages than can actually fit into the available physical memory (after taking into account what the OS itself uses etc). And when the overall system memory demand grows too high it will just kill some arbitrary process. On Linux this is the infamous OOM killer that uses the "niceness" score to determine what to kill.

And yes, for the OOM killer there's very little you can do.

But an allocation failure (nullptr or bad_alloc) does not mean OOM condition is happening in the system.

None of that matters: what is your application going to do if it tries to allocate 3mb of data from your 2mb allocator?

This is the far more meaningful part of the original comment:

> and furthermore most code is not in a position to do anything other than crash in an OOM scenario

Given that (unlike a language such as Zig) Rust doesn’t use a variety of different allocator types within a given system, choosing to reliably panic with a reasonable message and stack/trace is a very reasonable mindset to have.

Since we're talking about SQLite, by far the most memory it allocates is for the page cache.

If some allocation fails, the error bubbles up until a safe place, where some pages can be dropped from the cache, and the operation that failed can be tried again.

All this requires is that bubbling up this specific error condition doesn't allocate. Which SQLite purportedly tests.

I'll note that this is not entirely dissimilar to a system where an allocation that can't be immediately satisfied triggers a full garbage collection cycle before an OOM is raised (and where some data might be held through soft/weak pointers and dropped under pressure), just implemented in library code.

Sure, and this is completely sensible to do in a library.

But that’s not the point: what can most applications do when SQLite tells them that it encountered a memory error and couldn’t complete the transaction?

Abort and report an error to the user. In a CLI this would be a panic/abort, and in a service that would usually be implemented as a panic handler (which also catches other errors) that attempts to return an error response.

In this context, who cares if it’s an OOM error or another fatal exception? The outcome is the same.

Of course that’s not universal, but it covers 99% of use cases.

The topic is whether Rust should be used to re-implement SQLite.

If SQLite fails to allocate memory for a string or blob, it bubbles up the error, frees some data, and maybe tries again.

Your app may be "hopeless" if the error bubbles up all the way to it, that's your choice, but SQLite may have already handled the error internally, retried, and given your answer without you noticing.

Or it may at least have rolled back your transaction cleanly, instead of immediately crashing at the point of the failed allocation. And although crashing should not corrupt your database, a clean rollback is much faster to recover from, even if your app then decides to crash.

Your app, e.g. an HTTP server, might decide to drop the request, maybe close that SQLite connection, and stay alive to handle other ongoing and new requests.

SQLite wants to be programmed in a language were a failed allocation doesn't crash, and unlike most other code, SQLite is actually tested for how it behaves when malloc fails.

In C++ it will throw an exception which you can catch, and then gracefully report that the operation exceeded limits and/or perform some fallback.

Historically, a lot of C code fails to handle memory allocation failure properly because checking malloc etc for null result is too much work — C code tends to calm that a lot.

Bjarne Stroustrup added exceptions to C++ in part so that you could write programs that easily recover when memory allocation fails - that was the original motivation for exceptions.

In this one way, rust is a step backwards towards C. I hope that rust comes up with a better story around this, because in some applications it does matter.

Reply on news.ycombinator.com