impl Drop for Foo {
fn drop(..) {
self.cleanup();
self.free_memory();
}
}
even if `self.cleanup()` panics. When that happens, `self.free_memory` will never be called, and memory will be leaked if `panic=unwind`, which is the default behavior. If there is a `catch_unwind` somewhere, these leaks will grow over time.The same code in C++:
~foo() {
this->cleanup();
this->free_memory();
}
does not have this issue, because if `this->cleanup()` unwinds, that unwind makes `~foo` abort, because `~foo` is `noexcept(true)` by default.In Rust, one cannot make `Drop::drop` be "nounwind", so to emulate C++'s behavior, one needs to write:
impl Drop for Foo {
fn drop(...) {
if let Err(_) = cath_unwind(|| {
self.cleanup(); self.free_memory();
}) {
abort();
}
}
}
or use a `DropGuard` to make sure that `self.free_memory()` is called if `self.cleanup` throws, etc.Destructors being fallible are very weird things. In C++, destructors are infallible (at least by default). Rust supports fallible destructors, with all its cost (every time an object is dropped, doing this can fail, adding another "return" point from your function), yet minimal or zero value.
At least, I couldn't find any examples in the Rust book / reference / rust-by-example that show in which situations unwinding from a destructor is a good thing to do.
Sort of; it's very rare. You don't really write free_memory, unless you're doing some very specific unsafe things.
> When that happens, `self.free_memory` will never be called, and memory will be leaked if `panic=unwind`, which is the default behavior.
It will be leaked if panic=unwind and if you use catch_panic. catch_panic is not used very much. Partially because panic=abort is also a valid semantic, and partially because if this Drop impl is called when panic happens, regardless of panic settings, you'll get an abort if self.cleanup() panics.
> Rust supports fallible destructors,
This isn't really true. Fallability in Rust is spelled "Result<T, E>", and drop does not return one. You're talking about a non-recoverable error, which, it is true, Rust does not give you tools to prevent, really. That's because it's very, very rarely used, because the whole intention is to end the current thread of execution.
> At least, I couldn't find any examples in the Rust book / reference / rust-by-example that show in which situations unwinding from a destructor is a good thing to do.
There isn't, which is why it isn't done, which is why your point confused me :)
However I'm not in favor of the proposed solution - if the docs.rs server aborted every time a thread panicked we would have a lot of outages!
Absolutely. But I do think the difference between "idiomatic" and "rarely done" is valuable, here. And as you said yourself, it's not clear that these leaks are caused by this kind of thing.
If someone saw Rust code leaking memory all the time due to catching panics, I'd want to know about it, because it's very contradictory to my own experience, and I think that's interesting.
That's not the case in C++ where "throw" is a keyword you are expected to use for error handling.
If you take this to a Java example, a panic is more like an "Error" and less like an "Exception". That is to say, in rust, if something panics it is a sign of a major bug that shouldn't have any option for recovery.
With all that said, the same concept exists in C++. Try doing a 1/0 in C++ and see what happens. You don't get some nice exception to catch and you can't add a "noexcept" clause to stop it from happening.
In the same original example, if you have that 1/0 error in the underlying method and you handle it instead of crashing by tying into the OS specific "div by zero" garbage. You to can create a memory leak in code thought to be safe.
It goes to show that you can't (and shouldn't) expect to recover from everything. Crashing, IMO, is usually far safer than trying to fix things up and move forward.
However, the real world is not "conceptually." I don't think there's any real data about how often they happen, but at least my experience is that tools I write in Rust rarely end up showing me panic output.
> That's not the case in C++ where "throw" is a keyword you are expected to use for error handling.
Yes, that's correct. The intended semantics of the two features are very, very different.
Where is the proposed solution?
Where and why does docs.rs unwind from destructors?
In order for that work, we'd have to have 0 panics - not just few, but none. That requires none of our code to panic, none of our dependencies to panic, and none of our uses of the standard library to panic. If you gave me a limited time frame to run the server - say a week - I think it's possible to make docs.rs that robust. However, for a server that's meant to run 24/7 for weeks on end, I just don't think that's realistic.
What `catch_unwind` lets us do is localize panics to a single web request or crate build instead of it affecting the whole server. Of course we don't want to have 500s for any user, but we _especially_ don't want the whole server to be down until a team member has time to ssh in and restart it manually.
Of course, after that there's whole question of whether making the server that robust is a good use of time in the first place. Is it better to fix a few 500s every week or to fix long-standing bugs? I don't think it's clear that the 500s are more important if they don't affect many users.
Basically crashing is the safe way to release all resources in a problematic situation, at the cost of terminating the process. It's easy when you don't have shared resources at all (Erlang's case), and harder with threads: one threads crashes and frees its resources, another tries to use a shared resource already deallocated. If this can be avoided, life becomes vastly easier, at the cost of higher resource consumption.
If you are using a thread, you better not be leaking anything. Otherwise you are "ulimits" resources away from putting your "main" process into a crashing loop, e.g., if you leak 1 file-descriptor per crash, then you can crash your thread < 1000 times.
Performance-wise, you are probably worse with threads as well. On linux, you can initialize a web-server on your main process, and spawn new processes by forking it. Forking isn't only pretty much instantaneous, your child process is initialized with the same state as the parent, so you instantaneously get a fully initialized web server (e.g. with multiple threads already started in your task pool, etc.).
This shows why it was a bad decision. It allows doing something in the language (unwinding from destructors), that a lot of code needs to protect against (e.g. all the standard library collections, all collections in general, all types that own a resource, etc.), for absolutely no added value (doing that isn't a useful thing to do).
Aborting the process if `Drop::drop` unwinds would have simplified both the language and the programs written in the language without downsides.
That doing that was a good idea was known (CERT C++ requires non-throwing destructors), and C++ actually made a backward incompatible change in C++11 to fix that (making destructors `noexcept(true)` by default, e.g., see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n316...).
You yourself said that web servers are a place where this is a good idea.
> That doing that was a good idea was known (CERT C++ requires non-throwing destructors), and C++ actually made a backward incompatible change in C++11 to fix that (making destructors `noexcept(true)` by default, e.g., see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n316...).
Again, throwing in C++ and panicking in Rust are used very differently idiomatically, and in practice, due to the double panic issue, as I mentioned, this is effectively the same for Rust. Yes in theory you can panic in Drop but it will often end in an abort, so doing it for some kind of recoverable problem makes nearly no sense.
I said that "panicking from Drop::drop is a bad idea, but that panicking is a good idea in general (e.g. in web servers)".
We are in agreement that panicking is a good idea in general, but you seem to be turning that around, arguing that "Panicking is a good idea in general, therefore panicking from Drop::drop is a good idea". One does not follow from the other.
If you believe that unwinding from Drop::drop is a good idea, enumerate the value this feature adds, its costs, and make a case about why this trade-off is worth it.
All languages with unwinding in the same space as Rust do not allow unwinding from destructors, because it adds no value, and adds significant costs. For example, C++ and D do not support it. C++ used to support it, like Rust, but considered the value it added as "negative" (that code deserved to be broken), and changed its semantics to forbid this by default. AFAICT, the same arguments that apply there, apply 1:1 to Rust.
In Rust, the cost of this feature is real, e.g., from basic impls like the impl of `Drop` for slices, reused by most collections, to pretty much every Drop impl of every type guarding an important resource in a module that uses unsafe (e.g. the many Iterator drop guards, etc.).
So I stand by my original claim: panicking from Drop::drop is a bad idea: there are no worthy use-cases for it, its costs are real, pervasive, and unnecessary, because in practice nobody does this, but every writer of unsafe code must write code to defend from it, which makes trickier code even trickier to write, read, and properly test.
C++ is the sane language here. Rust is not.
I don't believe it's a good idea, which I have repeated in this thread numerous times. You seem to keep implying that it's an often used and good idiom, and I keep saying that it's not, which is why it's not used. Most of your points are about unwinding generally, not unwinding from Drop. I think it's really confusing the issue here.
I don't really think this is productive. It seems we're in agreement about one thing, though I am very confused as to how. Let's just leave it at that.
I said that this is code that safe Rust allows you to write [0] - it's actually safe code that beginners could write by accident [1] - and therefore all unsafe Rust programmers need to actually write Rust code that defends against this happening: otherwise their safe Rust APIs over unsafe Rust code are unsound, because if someone were to write safe Rust programs like [1], their unsafe Rust code would exhibit undefined behavior (the standard library and the language itself being the prime examples of this [2]).
The claim that panicking from Drop::drop will often cause a double-drop or terminate the program assumes that all unsafe code is written by people that properly defend all their unsafe code against it happening. We agree that no reasonable programmer would purportedly write such Drop impls, so I doubt that. It suffices for execution to reach a path that isn't properly guarded for the program to exhibit UB, and at that point, there aren't any guarantees about double-drops panicking or aborts doing anything meaningful.
That is, we have a feature we agree on does not do anything useful, yet adds a cost to all unsafe code.
[0] As opposed to C++ or D, which do not allow you to write Drop::drop implementations that unwind/fail, and therefore their programmers do not need to write code to protect themselves from something that nobody does, just in case somebody actually ends up doing it by accident.
[1] As easy as:
struct S;
impl Drop for S {
fn drop(&mut self) { unimplemented!() }
}
[2] These safe code example are actually used by the standard library test suite to test, e.g., the collections. So it isn't just the cost of having to read/write unsafe code that protects against something that does not make sense doing, but also the cost of then adding and maintaining the tests for those code paths.This isn't really true. All Rust functions that can panic are fallible, independently of whether their return type is `Result` or not.
There are idioms to use `Result` for recoverable failures and panics for "harder-to-recover" failures, but that's about it, and people do use `catch_unwind` on `main` to make their web-servers live forever, and they are doing it right, IMO (this is one of the many justified usecases of `catch_unwind`).
If the errors would actually be non-recoverable, `catch_unwind` wouldn't need to exist.
Also, even if you think as panics as "errors that will kill this thread", your program has multiple threads, and resources leaked by one thread continues to be leaked (e.g. memory is not reclaimed by the OS until the whole process exits), so you still have a leak.
Yes, in the abstract sense of fallible, but not in the terminology of Rust, or how the features are used generally. Defaults and language matter, and we've seen the actual usage follow. I don't ever remember seeing a crate suggest catch_unwind for error handling.
> our program has multiple threads, and resources leaked by one thread continues to be leaked
Yep, my bad, I simply made a mistake here.
This isn't fully accurate. It is leaked if you use `panic=unwind` and the panic unwinds the function. At that point, the program is still running, but the memory is unreachable through a pointer in the program, and therefore leaked.
Whether you actually catch the panic afterwards doesn't matter. Not catching it will terminate the program, but then the program will be terminated with "leaked" memory, and tools like valgrind will report it as such.
An uncaught panic terminates the thread. If that's the main thread the process terminates, but on other threads it continues with leaked memory, even without `catch_unwind`.
> catch_panic is not used very much.
Are you sure about that? I'd expect most multi-threaded (web)servers to continue after a panic. Otherwise any bug causing a panic (common in my experience) would force a potentially expensive restart of the server.
Yep, my bad, I simply made a mistake here.
> Are you sure about that? I'd expect most multi-threaded (web)servers to continue after a panic.
The two main use-cases for panics seem to be:
1. web servers 2. FFI, since panic across the boundary is UB
On 1, well, this is actually a contentious point. In general, the web world has moved more and more to more and more disposable web servers. You have to be resilient to something killing your web serer process, so you need the infrastructure here anyway. Servers re-starting isn't very expensive because you have a bunch of them already, and they don't start serving requests until they're up. Of course, "defense in depth" is a good idea, so doing both makes more sense than just one, but you can't get away with just catching panics if you want a robust service. And, a lot of web servers are single threaded these days....
Regardless, while web services are a big market for Rust, they're only one thing that it does, so I still think of this as "not that popular." Maybe that's wrong :)
Plus if a webserver handles more than one request in parallel (affects both multi-threaded and node.js style async designs), you want to finish handling all the other requests before shutting down gracefully.
In my experience with C# even exceptions caused by bugs/failed assertions (which would map to panics in rust) almost never cause issues which require a server restart. It's a bit worse in Rust, since a GC collects memory more reliably in case of errors than Rust (but even in Rust leaks should rarely happen on panic)
Interestingly they picked the best possible case to showcase rust's strengths, where the free happens on drop. If it were a case where you needed strict control over where that memory came from and _when_ it gets freed, C would seem like a nice fit, but rust is great when you only care that the memory is gone once your variable's out of scope.
https://doc.rust-lang.org/edition-guide/rust-2018/error-hand...