Comment by CuriousCosmic | Hacker News Reader

Comment by CuriousCosmic | Hacker News Reader

Parent Full thread

RAII is almost zero cost abstraction and the only reason it isn't is because C++ is made out of landmines (UB and complex, dated ABI, etc).

If you can use references (and sometimes moves), you can produce code that zero cost abstracts for RAII but there are a bunch of arcane rules that you need to follow to guarantee this (and you'll need to double check the generated assembly to verify this).

Now if you need to interact with pointers, you'll need at minimum `unique_ptr` which is not zero cost. It's low cost but there are a bunch of reasons why it can almost never be truly equivalent to a regular pointer. You can get it there but you'll need to follow even more arcane rules and even then sometimes it'll just not be possible to get true zero cost unless you use a custom version of the stdlib and likely the compiler to force the build to disregard ABI safety/compat.

Compare this to Rust where UB is supposed to be non-existent outside `unsafe`, where the compiler far better understands when code is crossing a potential ABI boundary and what specific parts of that boundary need to be upheld, and just in general better understanding when two things can't actually interfere with each other vs when they can.

So rust isn't true zero cost abstraction either by any means but it is so much closer to zero cost than C++ and that's without the cognitive overhead that C++ brings with it.

> Landmines [..] arcane rules [..]

So you're (primarily?) referring to "cognitive overhead" rather than more usual meaning of overhead as in "runtime overhead". Not sure about the ABI differences and how this ties into problems and limitations of `std::unique_ptr`. I find it usually straight forward to write "assembly level optimal" code and rarely if ever have used raw pointers in C++14, C++17. Can you given an example?

Let me be clear, C++ is a great language but I fully concur that it takes great skill and resolve to consistently navigate about the landmines you allude to. Rust is much easier in that sense (though I don't agree UB is non-existent outside of `unsafe`). But in terms of runtime overhead, memory use and even source code size, Rust occasionally forces my hand considerably harder (e.g. `RC<RefCell<..>>` vs `std::shared_ptr<>`)) in the direction of performance compromise. Again, not a problem per se when balanced against easier to develop and maintain code but I would persist that C++'s RAII is closer to the metal than Rust's.

When it comes to the overhead issues of unique_ptr, I always think of this video here:

https://youtu.be/rHIkrotSwcc?t=1192

At 26:50 he gets into how this relates to ABI, and how 'fixing' some of this would require a pretty severe ABI change, as well as destructive move semantics, which would require a severe change to how C++ works.

I frankly don't understand C++ well enough to fully judge about all of this, but everything he says seems entirely solid, and showcases how unique_ptr is in fact not zero-cost, and can in fact be a significant performance trade-off when you really need that extra bit of performance.

> Rust occasionally forces my hand considerably harder (e.g. RC<RefCell<..>> vs std::shared_ptr<>)) in the direction of performance compromise. How so? Is RC<RefCell<T>> more expensive than std::shared_ptr<T>?

> So you're (primarily?) referring to "cognitive overhead" rather than more usual meaning of overhead as in "runtime overhead"

I'm referring to both. The issue is that all that complexity makes it very difficult for programmers to write code that the compiler is consistently able to optimise to (or close to) zero overhead. It can be done with enough work and care but for your average C++ developer, they aren't going to be seeing zero overhead for a lot of the things that C++ generally leads us to assume are zero overhead.

> Can you given an example?

To copy the link that TwentyPosts provided (which is coincidentally also the first thing that came to my mind, albeit at a different timestamp) https://youtu.be/rHIkrotSwcc?t=1054.

The issues with `unique_ptr` in particular are that:

- They force the compiler to generate exception code where maybe there isn't any necessary if you pass them into a function. You can elide this by going through and annotating things that you know can't except with noexcept however most people don't even care about noexcept in the first place and consider it line-noise.

- Using move references to unique_ptr instead of pointers adds overhead by forcing the compiler to generate extra memory reads.

- Passing by value should be zero cost but it'd require breaking basically every aspect of the C++ ABI to do so. Because of this you still get a double indirection which adds overhead.

So your average C++ programmer probably won't do #1 which adds the largest overhead and if you need to transfer ownership you have to do #2 or #3. It's only really zero overhead when you can't transfer ownership (`unique_ptr<T> const &`) and even then, now you are looking over your shoulder and maybe it gives you overhead on rare occasions.

> Let me be clear, C++ is a great language but I fully concur that it takes great skill and resolve to consistently navigate about the landmines you allude to.

Agreed 100%. However I think to navigate some of those landmines you can't get rid of them, you just have to walk around them.

> (though I don't agree UB is non-existent outside of `unsafe`)

I agree that it's not non-existent however the UB outside `unsafe` is assumed to be a bug in the language that needs to either be fixed or formalised as defined behavior.

> ...

I think if you successfully navigate all of the landmines, C++ RAII can outperform Rust where Rust falls short and generally otherwise go toe-to-toe with Rust. However I think Rust gives you most if not all of that overhead reduction for a massively lower cost than C++ for the same benefit.

This talk is just clickbaity, the complains are valid, but to then go and use it to claim that you can never have zero cost abstractions is just ....

For example, the ABI used by Linux already passes many structs in registers. Fixing it to also include unique_ptr will indeed "break every aspect of the C++ ABI", but that is not really an unsurmountable problem: Windows breaks C++ ABI on almost every other release (and so does almost any language other than C), Linux user-space has done so in the past, and Linux the kernel just doesn't care.

There is no overhead with exceptions when there are no exceptions (e.g., in the kernel). Also think about what it means for the compiler to need to generate exception code: if you were not using unique_ptr, wouldn't you have to make a conditional and free the pointer by hand in such scenario? If the answer is 'yes', then it is not really 'overhead'.

> It's only really zero overhead when you can't transfer ownership (`unique_ptr<T> const &`)

I think you have this backwards. Pass-by-value can be zero overhead (as in, you can probably think of an ABI which makes it zero overhead and is compliant). It is precisely pass-by-const-reference that cannot be zero overhead, because according to the language rules you are passing a reference to a unique_ptr (i.e. a pointer to a pointer). I'd hardly call that a hidden landmine, as that's what the & means, and I think the average C++ developer is familiar with the concept.

Then again, why would you try to pass a unique_ptr by reference is over me. There is really no ownership transfer whatsoever, you are just contaminating your API with unique_ptr unnecessarily.

In fact, most of the overheads mentioned just go away if you don't use unique_ptr in function signatures, which is hardly unmanageable mental complexity. In fact, in the proposal here by the Linux kernel, you also cannot pass unique_ptrs through functions; they are type erased.

> For example, the ABI used by Linux already passes structs in registers

It's a bit more nuanced I think - what I understood from that talk is that the "overhead" of a unique_ptr is attached to the inability for a compiler to pass the unique_ptr as a value through a register but will instead have to use the stack (memory).

That said, this nuance of using stack instead of registers doesn't apply in the general case but _only_ for the unique_ptrs holding an object with non-trivial copy-constructors or non-trivial destructors - so, non-PODs.

If we accept this nuance as a measurable cost in actual workloads where it really matters, this is again only the artifact of Itanium ABI and something C++ can only adhere to but cannot change. Any language that speaks to the system (OS) must do the same.

That said, among many other and much more complex code transformations that we do I can hardly see using push/pop instructions as a bottleneck - our CPUs indeed implement push/pop (micro)instructions through a dedicated HW circuitry meaning that they're always on the fastpath and we don't pay the price for them as we do for "regular" instructions.

This is why I call this talk a BS and I see it only as a way how Google wanted to prove that "break the world" is worthy under the disguise of zero-cost abstractions that are only "sometimes" zero because of "C++/ABI".

> It's a bit more nuanced I think

Yes, I have edited it a bit; I did not want to claim that unique_ptr is universally passed through the stack, just that it is not an insurmontable problem to do so.

> this is again only the artifact of Itanium ABI and something C++ can only adhere to but cannot change. Any language that speaks to the system (OS) must do the same.

This is not true, though. 1) you can change and evolve this ABI (it has happened in the past), 2) you can also have an ABI for the system libraries and an ABI for the rest (see Windows, or see how any language other than C works), and 3) since the problem would only be about non-PODs, which are pretty much C++ specific, the system ABI would not be affected at all.

I see, thanks for clarifying, especially on number (2) which I wasn't aware of. I always had an impression that the compiler ABI is somewhat tightly coupled with the system ABI and which is why I used them almost interchangeably.

So, Google premise is that they can provide alternative unique_ptr implementation in a way so that it breaks the C++ ABI without touching the Itanium ABI but with the benefit of unique_ptr fitting into the registers always rather than almost always? If so, I am not aware that they have shown any such implementation and its impact to their codebase?

You won't be compatible with the SysV/Itanium ABI (it does include C++ bits), but you can definitely do it. It just happens that you will probably not have any problem to call a C library from such a new ABI (incl. libc and the kernel) since you only changed non-POD behavior, and those C libraries won't use anything but PODs.

The fact that you can have some structs with non-trivial methods that are still classified as PODs means the ABI is already solving the hard problem (i.e. these methods won't really have a 'this' pointer), so the limitation for types like unique_ptr sounds like easy to remove in an experiment.

So ... it appears that LLVM actually did the necessary code changes - libc++ with [[clang::trivial_abi]] on std::unique_ptr guarantees that the std::unique_ptr is always going to be passed directly through a register: https://libcxx.llvm.org/DesignDocs/UniquePtrTrivialAbi.html

Caveats outlined by the document:

> Function definitions now require complete type T for parameters with type std::unique_ptr<T>. The following code will no longer compile.

> Lifetime of std::unique_ptr<> arguments end earlier (at the end of the callee’s body, rather than at the end of the full expression containing the call).

> Lifetime of local returned std::unique_ptr<> ends earlier.

So, switch to a new ABI alone requires (1) a lot of manual code plumbing, e.g. replacing all your std::unique_ptr with forward-declared types with corresponding header includes across the whole codebase and (2) more serious session of discovering and fixing new std::unique_ptr runtime behavior using ASAN.

Benefits of this ABI change measured on the Google-scale? This is what they have to say:

> Google has measured performance improvements of up to 1.6% on some large server macrobenchmarks, and a small reduction in binary sizes.

IMO pretty serious caveats for no obvious benefits, even at such a large scale as Google. Measuring 1.6% of runtime performance benefit on macro-scale is hard, if not impossible, to differentiate from measuring the noise.

Reply on news.ycombinator.com