Hacker News Reader

Comment by dullgiulio | Hacker News Reader

Very relevant, because containers are often under what basically is ulimit.

Not that you can do much when malloc fails...

> Not that you can do much when malloc fails...

Tell the user they can’t do that operation? Use on-disk storage instead? Re-use memory you already have (such as evicting a cache and taking the memory it already had allocated)? Abandon what you were doing if it was only an optimisation and wasn’t essential (such as allocating memory as part of a speculative execution)? Run a garbage collector and try again?

Lots of options available in some situations.

In practice i haven't seen any application to gracefully handle out of memory situations. Even back in Windows 3.x days when running out of memory was common, most applications simply hanged or crashed (which also took the entire GUI with them) and those that didn't often had a "no memory, kthxbye" dialog that shut down the application. A very few rare cases would try to display some "sorry, no memory, close some programs and try again" dialog but even those were hit and miss, depending on the situation.

Once 32bit hardware-based virtual memory entered the picture, every single application would just assume you have endless RAM - memory checks are reserved for "common cases" like trying to create a 100000x100000 image in a (32bit) image editor.

The reason for all that is simple: out of memory situations are stupidly and increasingly rare and in most cases where they can happen there isn't much you can do (e.g. what would you do if you run out of memory while making the fourth button in a toolbar?) and really in the 99% of the cases there might only be five people in the entire universe that will encounter such a case (two of them will tweet about it though and amass a lot of "lol, those garbage developers" retweets) so littering your codebase to keep those five people (and their Twitter followers) happy is not worth the effort. I mean, are you really going to put a "run out of memory" check after every toolbar button allocation? And what are you going to do if that fails? What can you do?

AFAIK some modern languages nowadays even assume memory allocations wont fail (and if they do they just terminate).

> what would you do if you run out of memory while making the fourth button in a toolbar?

Rollback what I’d created so far, evict caches, ask malloc to trim, try again, if that failed roll back again and ask the user to reduce the volume of application data open by closing views or documents or whatever before they try again.

But yeah it’s a lot of engineering.

So you meant to say: ignore the possibility of this happening because it is extremely unlikely, too complex to handle and your time is better spent fixing much ore serious bugs that the users report.

Silly engineering like what you suggest is a waste of time. Non-critical applications that are allowed to crash (every normal desktop app) would end up overengineered and unmaintainable. Embedded systems that must give guarantees about their reliability simplify system design drastically. One of the things going out the window first is dynamic memory allocation. It has too many failure modes.

> ask the user

You just failed to create a toolbar button, how are you going to ask a user do something if you already failed to create a tiny UI element?

I guess you pre-allocate these UI elements when you start your application.

What if malloc fails at that point because some other application (we're in a preemptive multitasking OS) decided to gobble up all RAM?

If your application fails at start-up, you don't need to worry about saving state and graceful failure.

Keep in mind that Windows 3.x wasn't really the pinnacle of computing in its days. PCs were relatively irrelevant compared to the systems everyone apart from a mom and pop shop ran their business on.

So the typical behavior you saw was a combined effect of physical constraints and lack of skill or care.

No it wasn't which is exactly why i brought it up (well that, and it was the most popular - despite its shortcomings).

Plus, you can do all the graceful malloc() failure handling that you want, but if the kernel hasn't got overcommit disabled, then your program can still exit uncleanly when you later on try to use some malloc()d memory that was overcomitted.

Exiting cleanly would mean trapping SIGBUS/SIGSEGV (can't remember which one is invoked) and being able to roll-back from just about any point in your code or libraries)

You need to ensure overcommit is disabled, but because that's a system-wide setting, your program can't easily force that on.

https://www.sqlite.org/malloc.html is a welcome contrast to the typical behavior you're describing, but I agree that such measures are rare. (and, not an application: though it provides the mechanisms for applications to follow its lead.)

> In practice i haven't seen any application to gracefully handle out of memory situations

I have seen code hit an allocation failure, bubble the error up the stack, which causes various pieces of code along the stack to free their heap allocations, all the while the process is able to log what is going and keep running.

I have even seen this happen when the allocation that failed was tiny, less than 100 bytes.

What is your definition of graceful in this case? Presumably there was a “business need” for the allocation and the software isn’t just allocating for fun. An allocation failure is a hard fault, it’s not possible to honor the business need so something expected to happen cannot happen. The program could segfault when the null pointer returned is written to, that is kind of crappy. It could report the error “hey we are out of memory and couldn’t do xyz” or better “hey we are out of memory and couldn’t do xyz, this is probably because of abc, maybe you can adjust the tuning.” Or it could be a completely temporal thing, maybe you flush some caches or something to free up some memory and reschedule the work.

I don’t know, maybe this is old school, but yes you should put a check on every toolbar allocation for two reason: one you can gracefully report why your app isn’t doing it’s job and two it is a very very slippery slope when you start ignoring those errors as a matter of practice

> It could report the error “hey we are out of memory and couldn’t do xyz” or better “hey we are out of memory and couldn’t do xyz, this is probably because of abc, maybe you can adjust the tuning.”

What if you cannot do that report because the reporting itself needs memory that can fail?

> but yes you should put a check on every toolbar allocation for two reason: one you can gracefully report why your app isn’t doing it’s job

This is a reason to put checks in every toolbar allocation, but on the other hand there is also a reason to not do that: you are trying to catch a case that will only happen at a 0.00001% of the time yet to do that you are making the codebase more complex which will affect working with it 100% of the time.

Unless you are working in something like a medical or nuclear device, it is not worth to bother with such things. Even then when it comes to high risk programs relying on the programmer doing things right outside the core functionality is probably (i haven't worked on such projects myself so i'm guessing here) not a good idea and instead you should compartmentalize (e.g. the core functionality is running as a separate process from the UI and if the UI crashes, a watchdog restarts the UI process). But that is my guess here, i mainly have common end user/consumer applications in mind.

> Unless you are working in something like a medical or nuclear device, it is not worth to bother with such things.

Where "something like a medical or nuclear device" implies a piece of software that is expected to work correctly.

In other words, you should always ensure that failures resulting from malloc(3) returning an error (and all other errors) are suitably contained.

No, i did not implied that, please do not put words in my mouth.

My implication was software that if it fails it will kill people.

Otherwise if a program crashes because of a situation happens once every 100000000 runs, not only is worthless to worry about it, it actually is preferable to not do that as to keep the codebase clean and hence easier to maintain for bugs that actually do affect people.

Keeping the codebase clean is why I prefer that allocation errors throw an exception rather than returning NULL.

But then you have to worry about exception safety on every code which might call an allocating function, making the code more complex. This is worse on C++ where even innocent-looking code could be allocating memory in the middle of an operation (for instance, by calling a copy constructor which allocates).

> What if you cannot do that report because the reporting itself needs memory that can fail?

Allocate and reserve error-reporting memory early. For example, your ErrorReporter instance is initialized at start-up, and doesn't call malloc() when you make a report.

> What if you cannot do that report because the reporting itself needs memory that can fail?

Allocate a big block of memory at startup. Free it just before doing the reporting.

Depends; for example one video application I've written buffers as many incoming frames in memory as possible; if malloc fails when taking the buffer from 8000 to 9000 frames it's not a problem; just carry on running with the current buffer size.

If you write code with memory exhaustion in mind, which mostly means not relying on allocations in the error handling paths, malloc fails are perfectly fine to handle.

This is true, in my experience. If you keep your program fairly simple, implement it in plain C, avoid recursion, and keep these requirements in mind from the start, then it is possible to write a program that fails gracefully when it runs out of memory. Some old-school Unix daemons do this. But for most software it's impractical, so you might as well define your own xmalloc() that calls abort() if malloc() returns zero and at least have an immediate and obvious failure, rather than obscure memory leaks and data corruption.

An exception-safe C++ program can do the same through a std::bad_alloc exception.

If the out-of-memory was caused by something simple like accidentally loading 2G of data because of some odd data in an network request or a user selected file, and it rolls back to the start of the UI action or the incoming network request, the application may still work fine.

(I usually connected other out-of-resource conditions such as pipe/socket/open failures to std::bad_alloc too. They're pretty comparable in effect and give a bigger chance of actually exercising those exception paths)

> If the out-of-memory was caused by something simple like accidentally loading 2G of data because of some odd data in an network request or a user selected file, and it rolls back to the start of the UI action or the incoming network request, the application may still work fine.

If you have a system that handles independent requests, it might be more robust to model each request as its own process, and then simply allow those processes to crash if they find themselves in an unexpected condition (and handle process crashes).

It's certainly more robust, but a lot of overhead for what should be a very uncommon situation.

In eg an application server, requests themselves may be independent, but still share a lot of cached data - and as long as locking overhead doesn't overwhelm you, a single process is still the fastest way to share data between (worker) threads.

If that data is read-only then a fork()ed child process will share it just as efficiently as a thread would.

If your threads need to share interleaved modifications to some common state then that approach isn't viable. But in that case it will be very hard to "roll back"/"skip over" a failed request, as it's difficult to be confident that you haven't corrupted that shared data in a way that will cause the same problem for future requests.

fork only works if the central cache never updates. Eg a cache of compiled bytecode for scripts gets filled as those scripts are requested. A cache of database results fills as those results are requested.

I'm not sure why skipping over failed requests is hard. VMs associated with the request are aborted, the client gets a 500/503 and can retry later. Incomplete data doesn't get added to the caches at all, so no corruption of shared data.

It kind of goes both ways. If the data sharing pattern is simple then it's still quite simple to handle that at an inter-process level (e.g. perhaps you have to do cache updates via slower IPC, but if the cache updates only happen after the request has already been handled then they're not on the critical path. Or you might have the parent process handle the cache updates itself). If the data sharing pattern is complex and interleaved then it's hard to achieve with anything other than threads - but that's exactly the case that also makes it hard to avoid corrupting shared data.

You can use pool of processes, so you don't have to launch new process for every request. Also you can and should use shared memory for shared data. In the end it's almost like threads, but with more safety.

Unfortunately, that only works well in the absence of threads. With multiple threads, the out-of-memory could have been caused by code in another thread allocating too much.

When a memory control group runs out of space the process allocating just gets unilaterally killed with no notification. ulimit instead makes brk or mmap return a failure code, which the application could attempt to handle.

If you take a look at the Linux kernel, a hugely complex and sophisticated piece of software, you'll find that it gracefully handles allocation failures.

The kernel's "graceful handling" of allocation failures includes killing unrelated processes in an attempt to free some memory.

Also just deadlocking. The kernel's behavior in the face of real memory pressure is actually quite poor. It's unsafe to run a Linux box out of memory.

"graceful degradation" is one of the things worth trying.

You still have to gracefully shut down the application by closing file handles, aborting database transactions, etc.

The OS will close the file handles and any database worth its salt should also abort the transaction automatically when the connection to it is lost before the transaction is finished.

Reply on news.ycombinator.com