Hacker News Reader

Comment by iou | Hacker News Reader

Swap out the allocator https://users.rust-lang.org/t/optimizing-rust-binaries-obser...

Thanks! That really does seem to be the issue and I wouldn't have known about this, had I not asked. I will try this out and will update the blog post in ~8 hours time.

IME allocations is one of the main things making rust programs slow without diving into the more arcane stuff. So looking into unnecessary allocations and/or the performances of the allocator would be one of the first things to do (right after checking if you're compiling with optimisations).

Given your CPU graphs, and the large number of cores, I expect musl's allocator simply has very poor behaviour with respect to multithreading (e.g. limited or no threadlocal arenas, size-classing, etc…) leading to a lot of crosstalk, extreme contention on allocations, etc...

Tbh allocations (and related memory management things) are often the low hanging fruit in big picture optimization in many languages, incl c++.

Without familiarity with rust, I wasn't sure what they meant by "system allocator". Apparently that means libc's malloc. (Or HeapAlloc on Windows)

So I guess they statically link jemalloc but can optionally use libc malloc.

It's the other way around now, though the post that iou linked is from an earlier period when things worked differently. By default today, Rust programs use the same allocator that C programs use, which I think is provided by libc on Linux, and you have the option of using a custom allocator, like jemalloc. Historically however, all Rust programs used to use jemalloc by default.

I suppose there's no option to link statically against glibc because of the implications with LGPL (static linking triggers additional license clauses).

I think it's mainly that glibc has poor support for statically linking.

Ah, okay. Searched a bit, and it apparently requires you to find your own with dns client lookup library, avoid dlopen(), set GCONV_PATH, and so on.

For a long time glibc was maintained by someone who was of the opinion that if you wanted to statically link 3rd party libraries, then you shouldn't be allowed to build binaries.

It's interesting because there is an opposite cult (I believe some prominent plan9 and go people are adherents) that believes dynamic linking should literally not be a thing. I even recall reading some essays that introduction of a dynamic linker was some kind of tragic downfall in Unix history.

I think the truth is that both have costs and benefits. Dynamic linking is good for security patches, memory and disk usage. But it creates new opportunities for problems, for example ABI breakage becomes more significant of a problem and needs a lot of care to avoid. People distributing code, be it to end users or app stores or on servers with things like chroots, jails and containers, need to carry their dependencies anyway negating some of the benefits.

Note also that while it probably is a net-win for security to use dynamic linking, it's not 100% a win. I've seen security vulnerabilities introduced because of upgrades to dependencies.

Well, and LD_LIBRARY_PATH type vulnerabilities.

In general, do not expect dlopen() to work in statically linked programs (in any libc).

A key phrase I found googling "rust system allocator" is "provided by the operating system". On Unix, a dynamically linked libc would certainly fit the bill.

On Windows the C runtime is technically provided by the compiler and its runtime, so even that would make sense why they use HeapAlloc from Win32. (It's also common on Windows for different DLLs in the same process to have separate, incompatible libc mallocs, so I guess it might solve that too.)

Also, it’s extremely common to leave libc dynamically linked for operational reasons. It’s my understanding that by default Go statically links everything but libc.

Go doesn't link libc at all, generally.

Not on Linux, with some exceptions (it wants to play nice with nsswitch, after all). It does/has to on macOS, especially if you want your binaries to work on more than a single system release.

On Linux it has avoided libc for nsswitch in the common case since 2015: https://github.com/golang/go/issues/10485

Golang has only very recently (in its lifetime) and grudgingly admitted that operating systems that provide ABI stability at the DSO layer, rather than the Syscall layer, exist. It has been Linux-first for most of its life, and on Linux syscalls are the ABI stability layer. Not so pretty much anywhere else.

On MacOS, Go links libSystem rather than libc. And this was new in 2018: https://golang.org/doc/go1.11#runtime despite having a Mac port since ~2012. Prior to that they ignored the system ABI stability layer and just did raw Darwin syscalls. They still do so on the BSDs, despite this explicitly not being the supported stability interface.

I stand corrected.

Sure. Wasn't suggesting it as the default. Just curious why it wasn't an option.

I always assumed that that’s done for portability reasons, since libc deals with a lot of os specific details.

Symbol versioning for backwards compatibility is only possible with dynamic libraries, for example. Static linking is of course always backwards compatible for a given binary artifact, but is significantly bloatier and makes it harder to apply security fixes.

I wonder why glibc isn't just using jemalloc when it seems to perform much better than what it currently has.

Under most workloads jemalloc will use much more memory than ptmalloc (glibc).

I had the opposite experience. Jemalloc uses a lot less (virtual) memory, especially with multithreaded applications. The glibc allocator wastes quite a lot of memory mappings from a thread and never cleans them up even if a thread only had a burst of allocations (there is an upstream bug open about, they don't consider it a leak)

I was about to say nobody cares about virtual memory usage, but I can see how those who are extremely sensitive to runtime performance might care about page table entries or VMAs. I think most people are more concerned about RSS.

Just wanted to add that I strongly suggest thinking about the correct allocator for your applications and never using "the system allocator". In production context there should never be "a system allocator" anyway. Uninstall libc in prod. Force yourself to make a rational choice of memory allcoator.

> Uninstall libc in prod.

When you start to miss the thrill of it :D

Reply on news.ycombinator.com