“Shared libraries are not a good thing in general”
lore.kernel.org
lore.kernel.org
However, there important advantage of using .so files instead of building into a big statically linked binary is NOT that a library can be shared between different programs! The important benefit is dynamic linking. The ability to change which library is being used without needing to rebuild the main program is really important. Maybe rebuilding the program isn't practical. Maybe rebuilding the program isn't possible because it's closed/proprietary or the source no longer exists. If the program is statically linked, then nothing can happen - either it works or it doesn't. With dynamic linking, changes are possible by changing which library is loaded. I needed to use this ability last week to work around a problem in a closed, proprietary program that is no longer supported. While the workaround was an ugly LD_LIBRARY_PATH hack, at least it got the program to run.
It's just not worth it. A binary is a closure over the entire source text of the program and its libraries. That's what CI tested, that's what the authors developed against, and that's what you should run. Randomly changing stuff because you think it's cool is just going to introduce bugs that nobody else on Earth can reproduce. Nobody does it because it absolutely never works. You hear about LD_PRELOAD, write some different implementation of printf that tacks on "OH COOL WOW" to every statement, and then never touch it again. Finally, dynamic loading isn't even the right surface for messing with the behavior of existing binaries. It has the notable limitation of not being able to change the behavior of the program itself; you can only change the results of library calls.
I never want to see a dynamic library again. They have made people's lives miserable for decades.
As an intern at a data science company, I noticed that a certain closed-source shared library we were using was calling the same trigonometric functions with the same arguments over and again. So, I tried to use LD_PRELOAD to "monkey-patch" libm with caching and that reduced the running time of the data pipeline by 90%.
>I never want to see a dynamic library again.
And replace it with what? Completely statically linked libraries? How much disk space would linux userland require if every binary was to be statically linked?
We’re moving to a world where every program is containerized or sandboxed. There is no more real “sharing” of libraries, everything gets “statically linked” in a Docker image anyway.
Imagine you have 10 programs statically linked against the same library. How many will be promptly updated? How do you, as a system admin, track which ones were not?
Why? You can have both e.g. /lib/libfoobar-1.0.4.so /lib/libfoobar-1.0.3.so
Usually you would have /lib/libfoobar-1.so -> /lib/libfoobar-1.0.4.so, but it doesn't prevent you from linking the problematic program directly with libfoobar-1.0.3
That's incidental and not by principle, isn't it? You should be able to have a multiple versions of a library installed, and the dynamic linker would pick the appropriate version.
Heck you could even have multiple versions in the same process (if you depend on A and B, and B uses an older version of A internally). Unfortunately I think the current dynamic linker on Linux has a shared namespace for symbols.
I also disagree that it is not useful to be able to update libraries separately. The example that I like to give is a PyGtk app written against an earlier Gtk2. Without modifications, it ran against the very last Gtk2, but now supporting: Newest themes, draggable menubars, better IME, window size grippers, and all kinds of other improvements to Gtk.
In the days of ActiveX (and I realize this is a bad example because in reality you often had breakage), you could update a control and suddenly your app would have new functionality (like being able to read newer file format versions or more options to sort your grids and so on).
I've not experienced that. If the package is maintained at all, then it will get updated. People are still somehow maintaining Perl 5 libraries and putting out bugfix releases, which are functionally equivalent to dynamic linking.
If it's not maintained, then of course it will get removed from the distribution, for that and for a number of other reasons.
Then what is? Dropping a custom library next to a binary tends to be way easier than modifying the binary.
The distributions will take all front-line bug reports and the maintainer will forward them to upstream only if applicable. Or at least, that's how Debian operates.
It really isn't. At all. For example, let's take a notoriously egregious example of C++, which doesn't really have a stable ABI, and a major framework like Qt, which provides pretty much the whole platform. Since the introduction of Qt5, you can pretty much upgrade even between LTS releases without risking any problem: replace libs built with the same compiler tool chain, and you're ddone
This is not an isolated case. Shared libraries and semantic versioning is not a major technical hurdle, and one which in practice only has upsides.
It's not true at all. Libfoobar-1.0.4 is backward compatible with libfoobar-1.0.3, so both programs will work. If one of the programs or the lib has a bug, then it's easier to just fix the bug than to add burden on maintainers.
Moreover, both libfoobar.so.1.0.3 and libfoobar.so.1.0.4 can be installed at the same time, e.g. libfoobar.so.1.0.4 can be packaged as libfoobar-1.0.4, while libfoobar.so.1.0.3 can be packaged as compat-libfoobar-1.0.3.
If you have more questions, try to find answers in packaging guidelines for your distribution, or ask maintainers for help with packaging of your app.
- A whole lot of software is now open and freely available
- The code-changes-to-binary-updates pipeline has been greased by a couple orders of magnitude, from internet availability to CI to user expectations around automatic updates (for both commercial and non-commercial software)
- Disk space, and arguably even RAM, have become very cheap by comparison
Given these, it seems worthwhile to re-evaluate the tradeoffs that come with dynamic linking
Perhaps this is just a Linux distro thing, but as someone who closely monitors the Void packages repo, dynamic linking burdens distro maintainers with hundreds of additional packages which need to be individually tracked, tested and updated.
(Packages which could otherwise be vendored by upstream and statically linked during the build.)
Dynamic linking also adds complexity to distro upgrades, because dependant packages often need to be rebuilt when the libraries they dynamically link to are changed or upgraded. For example, Void’s recent switch from LibreSSL to OpenSSL borked my install script which wasn’t aware of the change. It resulted in a built system whose XBPS package manager couldn’t verify SSL certs. On Arch, AUR packages were notoriously prone to dynamic linking errors (back when I still used it).
Personally, I don’t find the bandwidth efficiency and CVE arguments for dynamic linking to be all that convincing [1]:
Will security vulnerabilities in libraries that have been statically
linked cause large or unmanagable updates?
Findings: not really
Not including libc, the only libraries which had "critical" or "high"
severity vulnerabilities in 2019 which affected over 100 binaries on
my system were dbus, gnutls, cairo, libssh2, and curl. 265 binaries
were affected by the rest.
The total download cost to upgrade all binaries on my system which
were affected by CVEs in 2019 is 3.8 GiB. This is reduced to 1.0
GiB if you eliminate glibc.
It is also unknown if any of these vulnerabilities would have
been introduced after the last build date for a given statically
linked binary; if so that binary would not need to be updated. Many
vulnerabilities are also limited to a specific code path or use-case,
and binaries which do not invoke that code path in their dependencies
will not be affected. A process to ascertain this information in
the wake of a vulnerability could be automated.
And you have to consider some performance penalty when using shared libraries. First you have a time penalty when loading the executable, since first you have to run the interpreter (ld-linux) and then your actual code. But also for each function call you have to make an indirect jump into it.
Although, I have mixed feeling about containers, because I fully appreciate that Unix is the application and the software that runs on it are just the calculations. In that world, sure, a shared library makes sense the same way a standard library makes sense for a programming language. Thus, a container is “just” a packaged application.
Regardless, this concept is so out of the realm of “performance” that it’s worth noting that the idea of trying to reduce memory use or hard disk space is not a valid argument for shared libraries.
CPU frequency and CPU cache are remaining small, so smaller binaries, which fit the cache, are running faster, and use less energy, and wastes less resources overall.
> Given these, it seems worthwhile to re-evaluate the tradeoffs that come with dynamic linking
Create your own distribution and show us benefits of static linking.
Having spent many hours avoiding bugs caused by this anti-feature I have to disagree. The library linked almost always must be the one that the software was built against. Changing it out is not viable for the vast majority of programs and libraries.
Just as an example, there is no good technical reason why I shouldn't be able to distribute the same ELF binary to every distro all the time. Except the fact that distros routinely name their shared objects differently and I can't predict ahead of time what they will be, and I can't feasibly write a package for every package manager in existence. So I statically link the binary and provide a download, thereby eliminating this class of bug.
Despite the rants of free software advocates, this solution is the one preferred by my users.
This is security vulnerabilities. If your application depends on a common library that had a vulnerability, I can fix it without you having to recompile your app.
With GLibc or X libraries a vulnerability there would result essentially requiring reinstallation of the entire OS.
Oh, your app also works on every single kernel, every different version of external applications, and supports every file tree of every distro? Sounds like you added a crap-ton of portability to your app in order to support all those distros.
> I can't feasibly write a package for every package manager in existence.
But you could create 3 packages for the 3 most commonly used package managers of your users. Or 3 tarballs of your app compiled for 3 platforms using Docker for the build environment. Which would take about a day or two, and simultaneously provide testing of your app in your users' environments.
Except it isn't, at least not for open source.
Most libraries do not have stable ABI's, even for C there are may ways you can mess that up. Even "seemingly clear cut cases" like some libc implementations ran into accidental ABI breakage in the past.
And just because the ABI didn't change doesn't mean the code is compatible.
It's not seldom that open source libraries get bugs because dynamic linking is used to force them to be used with versions of dependencies which happen to be ABI compatible (enough) but don't actually work with it/have sub-tile bugs. It sometimes gets to a point where it's a major anoyence/problem for some open source projects.
Then there is the thing that LD_LIBRARY_PATH and similar are MAJOR security holes, and most systems would be best of to use hardening techniques to disable it (not to be confused with `/etc/ld.so.conf`).
Through yes without question for not properly maintained closed source programs it is helpful. But then for them things like container images being able to run older versions of linux (besides the kernel) in a sandbox can be an option, too. Through a less ergonomic one. And not applicable in all cases.
I do not consider LD_LIBRARY_PATH or LD_PRELOAD more a security hole than PATH itself.
There is two scenarios:
- you control exactly how your program launcher (environment variables, absolute path) andit's a non issue
- you do not control the environment properly and the everything is a security hole.
That's said: DT_RUNPATH and RPATH are however beautiful security holes. They allow to hardcode loading path in the binary itself even with a controlled environment.
And many build tools let garbage inside these path unfortunately (e.g /tmp/my_build_dir )
I think the mess we created in ABI space is one of the failures of our indistry.
The dependency hell thing seems to come up when libraries change their APIs and then you have to scramble. Last one I recall struggling with was libpng.so but there are plenty of others.
Yes, dynamic linking makes this process easier, but it makes it so easy that both users and distro maintainers regularly break software without even realising it (hence why Red Hat use 5-year old versions of everything).
These problems are almost purely caused by dynamic linking though. The devs release something that was tested on some old version of ubuntu and now on your new fedora, things work different and the program is broken. While with static linking, it all just works.
Even that could be env-related, so maybe it surfaced once you moved to a new Fedora release.
Basically, the problems are pretty much the same with either approach, one has a more complex runtime environment, other has a more complex upgrade story.
While I'd agree with Linus' take on shared libraries, I'd still say that "statically linked libraries are not a good thing in general either" (the stress is on general).
Or just having the option of loading the library at all. If you don’t need the functionality offered by libxyz, you’re not required to use it. One then has no end of language extensions that can be loaded into a generic interpreter to fit a script to whatever job they have at hand.
Edit: Linus touched on this as a last point:
> Or, for those very rare programs that end up dynamically loading rare modules at run-time - not at startup - because that's their extension model.
My question: is it actually rare?
Static libs are embedded into most object-file formats (e.g. ELF) as isolated compilation units. Could you not just replace the static library within the object-file at the linkage level, in about the same way that old "resource editors" could replace individual resources within object-files, or the same way that programs like mkvmerge can replace individual streams within their target media-container format?
Static vs. dynamic linking is just about whether the top-level symbol table of the executable can be statically precomputed for that linkage. It doesn't impede you from re-computing the symbol table. That's what the linker already does, every time it links two compilation units together!
It’s a neat idea, but one obvious downside is the potential need to relink many executables when a shared component changes.
Today I learned about Application Binary Interfaces.
(and thanks for the generally insightful comment)
Or put in a different way: the problem is the "shared" part, not the "dynamic linking" part. Instead of static linking, you can avoid the issue of library versions by just shipping your shared libraries along with your program[0] - and this way, users still retain the ability to swap out components if the need arises.
--
[0] - Well, you could, if you could rely on LD_LIBRARY_PATH to be set to "." by default. Windows is doing it right, IMO, by prioritizing program location over system folders when searching for DLLs to load.
But is it worth it though? Even "big" statically linked Rust programs are a few dozen MBs of executable (and not all of that is read-only, and even less will be shared with any other executable at any given time). With things like LTO identical source files can result in different machine code as well.
In the end it would be a lot of trouble for potentially very little gains.
At first I was annoyed that Rust defaulted to dynamic linking, it felt inefficient and overkill. But the more I think about it the less I can really justify doing it any other way, there are very few benefits to dynamic linking these days. The only one I can think of is overriding an application's behaviour through LD_PRELOAD, but few people know how to do that these days.
No; they're just different names for the same concept.
> Isn't it common in windows to use DLLs and simply pack in everything you need?
Yes.
> Is that not the best of both worlds minus a bit of ram and disk?
No; it's more like the worst of both worlds. When it is feasible, static linking is strictly superior to this approach. (You've pointed out that it may not be feasible in some cases in sibling comments, and I agree.)
Bundling shared libraries is kinda the worst of both worlds: not getting any sharing between application (/system wide security patches) but not getting the performance benefits of static linking either. The only good thing is that you could manually replace e.g. a vulnerable version of a library inside this bundle.
And yeah… Flatpak/AppImage/snap distribution is kinda like that Windows-style one.
Shared Objects (.so) are the Linux version of shared libaries.
I.e. they are an implementation detail, and if you don't use them as "shared library" (in general) then it won't really fall under this category.
Several times last week when I needed to use an old proprietary program that used several outdated libraries.
Every single time I want to run a program (usually a game) that has a runtime dependency on pulseaudio[1]. (apulse[2] usually works to translate the libpulse ABI back to ALSA).
One time I had to write my own version of a library that specifically emulated the ABI used by an important program from an outside vendor. Obviously this didn't "just work"; it required a couple weeks of work to write the new version. The point isn't that future versions of a library will magically "just work". With a dynamically linked dependency replacement is at least possible. If the program in question was statically linked, nothing could be changed.
The question isn't about which method is less work or easier to maintain. The question that matters in the long run is if you want the basic ability to modify program's dependencies in the future to fix an important problem?
[1] As of a few years ago, the stupid design decisions in pulseaudio make it make it highly incompatible with my needs. Just having it installed makes runtime linking issues even worse. It also add insane amounts of latency (>5ms is bad. >50ms is insane) by design.
Also, there are legitimate use cases for shlibs-as-plugins, such as ODBC.
I fully understand why languages like rust demand static linking (their compilation model doesn't allow dynamic linking). But once you encounter a C-API boundary, you can as well make that a dynamic C-ABI boundary.
One could reasonably view the kernel use of loadable modules as an example of this utility.
In 40 years in the field, I've never needed that. Every single time, we just emitted an entirely new build - because it's _much easier._
> Maybe rebuilding the program isn't practical. Maybe rebuilding the program isn't possible because it's closed/proprietary or the source no longer exists
These are edge cases.
Nearly all the time when we develop, we are making changes in an existing codebase.
For you, the application developer. For end users of old or proprietary software, replacing libraries is much more feasible.
So what libraries have this property? I've read in old threads about even glibc making changes that break binary interface compatibility.
I get the feeling not much attention is paid to binary interfaces in the free software world.
You can request different ABI versions from glibc. And glibc doesn’t introduce breaking changes every day but rather every few years.
I disagree. The main value proposition of shared libraries is being able to upgrade them without requiring to rebuild the application. Sure, libraries need to be competently maintained to ensure that patch releases are indeed compatible, but that still opens the door to fixing vulnerabilities on the fly by simply upgrading the lib that sneaks in the vulnerability, which shared libs allow even to end-users by simply dropping in a file.
One of my favorite libraries is SDL. Of course, when SDL2 came out, they moved away from the LGPL license to a more liberal zlib license, which allows static linking without infecting the license of your project. Great, except now as years go by and games are abandoned, they stop working in new environments, or at least don't work as well, and you can't just force in a new dynamic version of SDL to save things like you used to be able to.
The solution they came up with: even if the program is static linked, the user can define an environmental variable to point to a dynamic lib and that will be used instead. Best of both worlds, at least going forward for programs that use a version of SDL2 that incorporates the feature. (Some more details: https://www.reddit.com/r/linux_gaming/comments/1upn39/sdl2_a...)
I personally think with the stalling of serial speed in CPUs, we will eventually have to optimize and stabilize interfaces.
I think some of that will require stabilization of many APIs and libraries such that they don't change over the span of a decade.
This era is basically unthinkable in the current age, but I think that is because we have been living in the "free lunch" era of faster better CPUs every year for too long.
Much like all of the 20th century was an era of resource availability. The 21st will be an era of succeeding within resource constraints: with efficiency.
Those libraries will be excellent candidates for shared libraries.
A middle ground could be distros can keep control over the version that's being statically linked at build time. It would do away with the dependency conflicts that happen now and result in a better user experience. The app/libraries can be updated as needed together. This is what's happening in a venv or container too, once you've built the venv or container it's effectively static.
You do need to keep track of the library versions ultimately at build time. This effectively means handing things off to the build system, and distros like Nix have effectively wrapped build systems to some success I think.
But for most mature build systems we largely just use the toolchain as-is. CMake, pkg-config, autotools, and other toolchains or technologies generally allow enough flexibility to enable non-FHS builds to be done in a consistent and easy manner. Actually, creating nix packages for upstreams which have good release management discipline is trivial once you're familiar with nix.
What makes nix special is that you when you go to build the package, you reference the dependencies through their canonical name, but nix translates these dependencies to a unique paths which are hashed to capture all inputs (including configuration flags) used to create the dependency. So you can use multiple conflicting versions of a dependency in a package's dependency tree without issue. Dependencies can be shared if a dependency matches exactly, but can differ if needed. And that's something that FHS distros cannot do.
Actually, since nix never makes any assumptions about what's installed on the system, you're free to use it on any distro. And it's hermetic, so the host system wouldn't be changed outside of the `/nix` directory. You can even use is it even on macOS, although it's not as well supported, but should work fine for common packages.
> A middle ground could be distros can keep control over the version that's being statically linked at build time.
Nixpkgs already does this, even if the dependency is dynamically linked. So there's no additional overhead with switching it out to the static variants; other than upstreams may not support that scenario as well as "traditional" dynamic linking.
The theory is as follows: If a security flaw (or other serious bug) is found, then you can fix it centrally if you use shared libraries, as opposed to finding every application that uses the library and updating each separately.
In practice this doesn't work because each application has to be tested against the new version of the library.
This comes up a lot, but how often do you end up in a scenario where there's a critical security hole and you _can't_ patch it because one program somewhere is incompatible with the new version? Maybe even a program that isn't security critical.
Plus what you mentioned about testing. If you update each application individually, you can do it piecemeal. You can test your critical security software and roll out a new version immediately, and then you can test the less critical software second. In some systems you can also do partial testing, and then do full testing later because you're confident that you have the ability to roll back just one or two pieces of software if they break.
It's the same amount of work, but you don't have to wait for literally the entire system to be stable before you start rolling out patches.
I don't think it's 100% a bad idea to use shared libraries, I do think there are some instances where it does make sense, and centralized interfaces/dependencies have some advantages, particularly for very core, obviously shared, extremely stable interfaces like Linus is talking about here. But it's not like these are strict advantages and in many instances I suspect that central fixes can actually be downsides. You don't want to be waiting on your entire system to get tested before you roll out a security fix for a dependency in your web server.
Talking about user usecases, every time I play a game on Steam. At the very least there are GNU TLS versioning problems. That's why steam packages it's own library system containing multiple versions of the same library -- thus rendering all of the benefits of shared libraries completely null.
One day, game developers will package static binaries, and the compiler will be able to rip out all of the functions that they don't use, and I won't have to have 5 - 20 copies of the same library on my system just sitting around -- worse if you have multiple steam installs in the same /home partition because you're a distro hopper.
One day...
These are such different use cases that I think completely different standards and processes as well as build systems are going to become the norm for big critical infrastructure versus what is running on your favorite laptop.
Link-time optimization is already a thing. It's sad it doesn't seem to be used more often.
Wouldn't you still need at least runtime linking (dlopen) to link to OpenGL/Vulkan/etc.?
That’s not the point. The point is having to find and patch multiple copies of a library in case of vulnerability instead of just one.
Giving up the policy to enforce shared libraries would just make the work of security teams much harder.
But it doesn't really matter. What matters is that whatever system is in use needs to have a control system that can quickly and reliably tell you everything that uses a vulnerable library version, and can then apply the fixes where available and remind you of the deficiencies.
That metadata and dependency checking can be handled in any number of ways, but if it's not done, you are guaranteed not to know what's going on.
If a library is used inside a unikernel, inside a container, inside a virtual machine, inside venvs or stows or pex's or bundles, the person responsible for runtime operations needs to be able to ask what is using this library, and what version, and how can it be patched. Getting an incomplete answer is bad.
But there's one other advantage of the shared library thing, which is that when you need to react fast (to a critical security vulnerability, for example), it is possible to do it without coordinating with N number of project/package maintainers and getting them all to rebuild.
You still do want to coordinate (at least for testing purposes), but maybe in emergencies it's more important to get a fix in place ASAP.
...and then you deploy and discover that somebody was depending on that "unwanted" side effect.
I fully agree with that, didn't understand the rest.
Now it's a month later. What do you need in ongoing operations, assuming you want to keep providing reasonable security?
You need to know when any of your dependencies makes a security-related change, and then you need to evaluate whether it affects you.
You need to know which systems in your service are running which versions of that dependency.
You need to be able to test the new version.
You need to be able to deploy the new version.
It doesn't matter what your underlying paradigm is. Microservices, unikernels, "serverless", monoliths, packages, virtual machines, containers, Kubernetes, OpenStack, blah blah blah. Whatever you've got, it needs to fulfill those functions in a way which is effective and efficient.
The problem is that relatively few such systems do, and of those that do, some of them don't cooperate well with each other.
It's plausible that you have operating system packages with a great upstream security team, so you get new releases promptly... and at the same time, you use a bunch of Python Packages that are not packaged by your OS, so you need to subscribe individually to each of their changefeeds and pay attention.
Does that help?
The whole "shared libraries are better for security" idea is basically stuck in a world where we don't really have proper dependency management and everyone just depends on "version whatever".
This is interestingly also holding back a lot of ABI breaking changes in C++ because people are afraid it'll break the world... which, tbf, it will in a shared library world. If dependencies are managed properly... not so much.
I wish distros could just go back to managing applications rather than worrying so much about libraries.
EDIT: There are advantages with deduplicating the memory for identical libraries, etc., but I don't see that as a major concern in today's world unless you're working with seriously memory-constrained systems. Even just checksumming memory pages and deduplicating that way might be good enough.
And in case your definition of proper dependency management is 'stricter', then you simply state that you depend on a vulnerable version, and fixing the issue will be far more cumbersome as it requires manual intervention as well, instead of an automated update and rebuild.
If it is looser, then it will also be far more cumbersome, as you have to watch out for breakage when trying to rebuild, and you need to update your program for the new API of the library before you can even fix the issue at all.
In practice it works: see autopkgtest and https://ci.debian.net, which reruns the tests for each reverse dependency of a package, every time a library gets updated.
I know for a fact that other commercial, corporate-backed distributions are far away from that sophistication, though.
No, they're not. Both Fedora/RHEL and openSUSE/SLE do similar checks. Every update submission goes through these kinds of checks.
Heck, the guy who created autopkgtest works at Red Hat and helped design the testing system used in Fedora and RHEL now. openSUSE has been doing dependency checks and tests for almost a decade now, with the combination of the openSUSE Build Service and OpenQA.
No, rebuilding is way heavier.
Even worse, some languages insist on building against fixed version of their dependencies.
It forces distributions to keep multiple version of the same library and this leads to a combinatorics explosion of packages to fix, backport, compile and test.
It's simply unsustainable and it's hurting distributions already.
It's worse if the program that needs the update also is broken by the update, of course.
Now it's your choice... you either lose B but protect the rest of the infrastructure from hackers... or you think the CVE doesn't apply to your usecase (internal thing on a testing server), and don't upgrade L to keep B working.
You can also install both versions of L. You can also patch the broken part of L out at the old version, if it's not mission critical. There's a lot of things you can do.
Having one giant binary file with everything statically compiled in is worse in every way, except for distribution-as-a-single-file (but you can already do this now, by putting the binary and the libraries in a single zip, dump everytinh in /opt/foo, and let user find the vulnerable library manually... which again, sucks.
Now with Docker, it becomes, find all the Docker containers that have that shared library, which may even be a bigger pain.
Not all uses, but it seems as if a significant use of Docker is just a a very complicated replacement for a single statically linked binary.
Imagine statically linking openssl having to rebuild tenths of thousands of packages every time there's an update!
Debian security issues thousands patched shared libraries every year without testing them against every program, and without causing failures. They do that by back porting the security fix to the version of the library Debian uses.
I gather you are a developer (as am I), and I'm guessing that scenario didn't occur to you as no developer would do it. But without it Debian possibly wouldn't be sustainable. There are 21,000 packages linked against libc in Debian. Recompiling all of them when a security problem happens in libc may well be beyond the resources Debian has available.
In fact while it's true backward compatibility can't be trusted for some libraries, it can for many. That's easily verified - Debian ships about 70,000 packages, yet typically Debian ships just one version of each library. Again the stand out example is libc - which is why Debian can fearlessly link 21,000 packages again the same version.
I'm guess most of the people here criticising shared libraries are developers, and it's true shared libraries don't offer application developers much. But they weren't created by application developers or for application developers. They were created by the likes of SUN and Microsoft, would wanted to skip one WIN32.DLL so they could update just that when they found a defect in it. In Microsoft's case recompiling every program that depended on it was literally impossible.
On Linux it the only thing it should change is that if a vulnerability is discovered let's say in OpenSSL all the software that depends on OpenSSL must be updated, and that could be potentially half the binaries of your system. It's a matter of download size, that in reality doesn't matter that much (and in theory can be optimized with package managers that applies binary patches to packages).
It's the maintainers of the distribution that notes the vulnerability in OpenSSL and decides to rebuild all packages that are statically linked to the vulnerable library. But for the final user the only thing he has to do is still an `apt upgrade` or `pacman -Syu` or whatever, and he would still get all the software patched.
Most distros dislike doing so, and scale it back as soon as it becomes technically possible.
For overlapping memory, use memmove()
Did glibc change the behavior of memcpy (the ABI symbol) or memcpy (the C API symbol) which currently maps to the memcpy@GLIBC_2.14 ABI symbol?
- shared libs
- name indirection (dns domains, css classes)
The kernel moves fast, because it has to. The usermode does not. The libc interface and relevant syscalls are backwards compatible to a greater degree than would ever be reasonable with kernel drivers. Windows does the best here arguably and drivers still had to break a few times in the past couple decades — not All drivers, but certainly some.
I guess the fuller picture to me would need to acknowledge the difference between modular plugins, library dependencies, and programs, as well as the difference between kernel and userland. The latter is definitely important: it’s very rare in userland that your CPU vendor matters over just the ISA itself, but in privileged/supervisor mode, often there are more differences directly exposed, as one example of potential pitfall. I think this sortof exemplifies the increased responsibility of the kernel to keep drivers working through refactors and changing hardware, especially on Linux where the hardware support is extremely diverse.
I do still agree with Linus regarding dynamic libraries, but I also think the two cases make sense for different reasons.
By a light-year.
> and drivers still had to break a few times in the past couple decades
Like 2 times in the past, what, 20 years?
Linux doesn’t attempt to provide a stable driver model, but based on the fact that many users are stuck on old kernels due to unmaintained drivers, I’m not sure if it would really help that much if it did. Desktop PCs are the place where drivers are most likely to be maintained long term and thats a small part of the Linux install base. The dynamics differ more in the wide range of supported devices.
If I don't use any features that require libbz2 or whatever a piece of software shouldn't complain if libbz2 isn't there on the system. If I add it to the system I should be able to use those features without recompiling and relinking everything.
Half of the features of autoconf such as "is this library with this method signature available?" should be moved into the dynamic linker.
Software would need to be rewritten to be more lazy and modular, but it'd make dynamic shared libraries actually dynamic. Having to make all those decisions at compile time instead of run time isn't very dynamic.
For different apps to use different versions of dependencies but still interact with each other, they would need strict rules about how to use interfaces to different versions. You obviously don't want two different programs that use two different versions of a library to talk to one another, because what if version A has a different schema than version B? Unless there was a very specific way to pass along interface information between two programs, so that they could independently handle changes in their interfaces.
This does exist--see the Vulkan API, for example.
And mostly everybody ignores it. Most programmers effectively write a wrapper to load everything and then treat it like the dynamic linker pulled everything in.
The alternative is when the functionality isn't core. We call those plugins and people use them quite a bit.
The problem is now you get a lot of bug reports of "Feature <X> doesn't work." "Well you don't have plugin <Y>. Closed."
Static linkage is goodness.
And, I'm tired of the dynamic library people beating the "security upgrade" dead horse. The biggest problem with the "security upgrade" argument is that a significant number of people refuse to upgrade because an upgrade always breaks something because everything is dynamically linked.
If everything was statically linked, their pet program wouldn't break and they'd be more likely to upgrade.
I’ve felt this with Linus, Dave Cutler and other “10x” engineers. Their code really isn’t that much better than anyone else with similar level of experience. Lot of engineers come up with neat solutions. But listening to these men or reading something they’ve written on their topic of interest, it feels as if you’d figured out the thing they’d just explained by yourself.
Nor even his second; I'm fairly sure it's his third (at least chronologically).
Why? As a Finland-Swede -- one of the Swedish-speaking minority in Finland -- he would have learnt Swedish as his mother tongue at home (and probably kindergarten / pre-school, if he went to any), and Finnish as his second, at the latest in elementary school. Perhaps in kindergarten / pre-school -- bilingual ones were rarer in his day, but some people put their kids in that of the "opposite" language explicitly to give them a head start on that. Then in around second to fourth grade he would have taken his first foreign language, which for an overwhelming majority is English (though it could also be German or French or something, and nowadays even more alternatives, like Spanish and Russian and whatnot, are available at least at central schools in larger cities). After that he may well have taken another foreign language in parallel, I have no idea (in primary school, up to and including grade 9, that is). I think in his school days, the 1970s / early 80s, that would statistically most likely have been German.
English being his third in stead of second language is not necessarily a disadvantage; on the contrary, I suspect it's actually an advantage for him to be at least tri- in stead of just bilingual: Knowing more languages in and of itself may make you more eloquent in all of them.
Not knowing him personally[1], so wild-ass guess: I'd tend to think he regards English as his second-but-close-to-first language now, having lived in the USA for so long. Probably still thinks -- to the extent one does that in any language at all -- and dreams in Swedish, and speaks it with his wife, but perhaps at least as much English with his kids who are born there.
At least that's how it works for me. Disclosure: second-hand knowledge of the Finnish school system by having a kid in it, first-hand experience of immigration twice over.
___ [1]: Met him once, about twenty years ago, when he signed a copy of Just for Fun for me.
"every program the system runs depends on library foo, so why is this loaded into memory x times? Let's load it once and share it"
"there are 10 versions of library foo, none of them are backwardly compatible. Every application loads its own version of foo. There are now at least 5 versions of foo in memory at once. Why is the OS trying to manage this? We should statically link every application and stop worrying about it"
This has already gone back and forth at least once since I started working in the industry.
What about when a security vulnerability in a shared library is found? Each dependent app would need to be explicitly changed to accept the patched version, once compatibility has been established. The problem with trying to force-update all apps by patching the sole version of a shared dependency is that it may break some apps. This runs afoul of the maxim "security at the expense of usability comes at the expense of security." At best, you end up with some hacky dependency-management system, at worst the patch remains all-or-nothing so that the slowest apps veto the update.
You can also imagine that the list of versions of a shared library could be maintained in an implicitly declarative fashion, so that old versions with no dependent apps could be garbage-collected using reference-counting.
I think the big reason Linux has mostly dodged "DLL hell" is that a large chunk of Linux applications are either built from source on the target system, or installed via packages that were built specifically for the target system.
However, if you try to build an application binary, then ship it on more than a handful of Linux distros (and even versions within distros), you'll quickly start running into issues with GLIBC symbols etc.
Containers makes this incredibly easy as you're basically shipping exactly what you built your code with.
I think this is also a place where languages with package managers & the corresponding ecosystems really shine (e.g. Golang, Rust etc). These do well because they are building from source against runtimes that don't involve a pile of shared libraries as you would be doing in C/C++.
Correspondingly, that's a non-technical thing to appreciate about shared libraries: they ended up creating the environment where building programs from source or installing them from maintainer provided packages was the only way to do things. If we end up moving to languages where static linking is the norm (or the only option), ecosystem shift away from the maintainer + source based approach might be unavoidable.
To my mind, that's pretty clearly a bad thing. It's how you end up with the node ecosystem's security problems, not something as reliable as most Linux desktops have been for decades.
So they're usually not shared, but merely use the shared library file format. For a practical example, most software on Windows will bring its own folder with DLLs.
One step further, this is probably why we're now seeing docker everywhere: it's a Linux equivalent to Windows and Mac app store apps.
I dare say this also adds credit to the approach of FreeBSD, where the base system itself includes some basic libraries such as openssl, etc. Updating the system patches the libraries uniformly, so any ports using them are automatically patched. And there is no problem with security patches breaking compatibility, because the version of libraries in base remains mostly stable for a particular version of FreeBSD and security patches are backported as necessary. Which is of course what happens in popular long-term support Linux distributions too, albeit at the package level.
It is also possible to have ports that use a different version of a library too, though that often requires building custom packages locally or via a centralized Poudriere builder.
There is a different market called embedded devices, 64MB-256MB RAM are still very common and there are probably 100x more devices in this segment than LF members' fat ones combined. Without shared library, you will quickly run out of RAM/Storage space on them.
Just like Linus for PC, I think it sensible to consider libc and other to be dynamically linked also on embedded systems, but maybe even less for other software.
Today with the cheap memory resources and fast interfacing you are often better off with programs that are statically linked as they have no run dependencies other than the OS.
I'm not sure that I would want all of Windows libraries to be statically linked to all my applications.
Sometimes you want static linking - sometimes you don't.
Also, Linus talks about this sort of exception in his post: "unless it's some very core library used by a lot of things (ie particularly things like GUI libraries like gnome or Qt or similar)".
This is a very very tiny part of an app. UI libraries with prebuilt controls are already almost always bundled with apps.
The problem with Linux also have to do with license, giving GPL can get some program into trouble by statically linking GPL code.
There are 3 cases for me that can force one to use shared libraries: Licensing, Dynamic loading a DSO at runtime (like in Apache modules) and when you are developing a big project, but once you need to ship the production code, get it back to statically linking everything.
Other from that i don't see why you would want this.
Why not?
> Sometimes you want static linking - sometimes you don't.
Why wouldn't you here?
This might be the case for developers, and for some people in the first world, but much of the world is using older hardware with relatively smaller amounts of memory and storage.
I've noticed developers tend to buy new computers often, but a lot of people just have one computer they bought several years ago and have no reason to upgrade it because it works well enough for Facebook, YouTube and email.
If you read Linus' email, you'll see that for most libraries it doesn't save any memory or disk space.
What it WOULD do to link statically is save the extra IO needed to load shared libraries (somewhat, the same code would be loaded at program execution instead of dynamically).
And the memory savings are often actually negative
And the dynamic linking will actually cause a lot _less_ sharing because of all the fixups.
If that library is statically linked, that one function is compiled into your program. The rest of the library is ignored.
If that library is dynamically linked, the entire library is loaded into memory, and the function address is shared with your program. More memory used.
The advantage of dynamic linking is if that specific version of the library is already in memory (because another application is using it) then it takes up no additional memory. Less memory used.
False. Shared library pages, like pages in executables, are loaded only on demand. You pay a memory cost for what you use, not what's available.
At least for gcc, this isn't true unless you pass some obscure flags in; the whole .a gets pulled in
If you consider inlining and dead code elimination from inter-procedural optimizations, this could sometime offset the savings from a library shared across 2~5 binaries.
(Also, the fixups themselves could actually be different between different processes due to address space layout randomization.)
I get this now. The fixups are runtime patches for call addresses in the application code to point to where the shared library function was mmap()'d. Those patches, although they will all be the same, are in fact COW writes which will in turn prevent page sharing. The solution that Linus doesn't mention is prelink.
I firmly believe that all applications should ship their dependencies. Launching an application should never be more complication than “unzip and double-click executable”.
Docker only exists because run-time environments are too complicated, convoluted, and mutually incompatible. It’s insane.
> Docker containers are kind of neat. They are also kind of a craven surrender to the rotting mess of excessive software complexity. - John Carmack (https://mobile.twitter.com/id_aa_carmack/status/138510311097...)
Let's suppose this happens. I'm wondering how all those Python, Ruby, Lua and whatnot programs would use database drivers, xml parsers, protocol interfaces, GUI frameworks, etc - all that stuff they are currently loading as shared libs? What are the alternatives?
2. How am I supposed to load a shared object if there are no shared libraries anymore?
This is literally the question I've asked: is there any other mechanism for achieving 2. other than shared libs?
Even if we look at security it in my experience doesn't help much in practice (besides the same exceptions of widely used system libraries). Instead it can even reduce security by introducing subtitle bugs due to unexpected interactions between the dependency and the dependent because the versions of both never had been meant to be used with each other.
And at least wrt. open source it's always possible to just recompile the software with patches.
Which brings me to another kind of software where shared libraries make sense: Proprietary likely to not get proper maintained software with non-proprietary dependencies.
If it is abandonware, then hoping that shared libraries will save you is at least naive.
Currently shared libraries only cause pain to the linux desktop users. I always postpone updating my system because I don't want to deal with the broken dependencies. It is insane how bad the situation is.
Technically speaking, would it be possible to design a Linux distro so that the installed executables are statically linked but an update for a library doesn't require redownloading large executables for every package that depends on the library?
https://www.chromium.org/developers/design-documents/softwar...
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=538339
It would still not be pleasant for disk space. Ideally I want my OS as well as the VMs to be small and require as little ram as possible. I don't want to update my entire OS just because glibc had a critical bug.
There are a number of ways to solve the latter problem, though: dnf, for example, includes the ability to download only the diff between your installed RPM and the version you're upgrading to.
(I don't know enough about Gentoo to know if it already does.)
In response, Gentoo developed methods to ship binaries for the common cases.
Also security updates. Other than that, static libs for the win.
Other examples, speeding up "link" times - Chrome uses component builds - e.g. bundle a bunch of C++ code as .dll and compile/load only that, rather than relink the whole .exe - slower to start and use, but apparently allows you faster development.
For what we do, we are using .dlls to enforce that certain C++ code (libs) are not suddenly using symbol from another lib if this was not intended (granted this could've been done if we compiled single .exe for the said lib, but gets the point). We also have mode where we use these .dlls instead of them being statically linked (having both).
If linking has such a huge impact on Chromium dev cycles, maybe they should make their codebase less of a bloated hog :)
Linking all of Firefox into one libxul.so with LLD takes basically no time (on a good machine) for a release build. Slightly noticeable with all the debug info but still pretty quick.
IMO the best reason to use shared libraries has always been cases where updates are likely critical and there's a well-defined interface that is unlikely to have breaking changes. Like libssl. If there's a security fix, I'd love to have `apt update` apply that fix for everything, and not worry about verifying that everything that might have statically linked it gets updated too. Although ideally the latter would be easy anyway.
It's also nice for licensing - being able to have dependencies that you don't have to redistribute yourself.
There are some shared dll's too, perhaps most notably the OS APIs. But 99% of the dll's on my system sit in Programs\specificprogram\x.dll
"Dynamically linked", doesn't imply "Shared between multiple executables". The trend on windows just like on Linux and Mac is for bundled-everything even for the thing you'd think was last to get there on windows: things like C++ runtimes and .NET frameworks are now bundled with applications.
The tradeoff between patching in one place, and being forced to maintain backwards compatibility and effectively make the runtime an OS component was won hands down by the bundle everything strategy.
You can have DLL's bundled with your app, so that the app links at runtime, but really they're not for 'sharing' at the OS level so to speak.
I think the number of DLL's shared should be very small, and as you say 'very stable ABI' like for OS level things.
Everything else - in the bundle.
To give some historical perspective of why, check out this Wikipedia article on "DLL hell", more specifically the "DLL stomping" section:
While it is one failure mode, and a well-understood one, to end up finding that you've been owned by an application with a hidden dependency that had a security hole that you didn't know about, the "apt update" means that your ability to fix a security hole is now waiting on the time it takes you to update thousands or tends of thousands of programs, 99% of which you don't and never will use, against the One True Copy of the library.
It's important to avoid equating "Linux" with "{Desktop,Server} {Ubuntu,Debian,Fedora,SUSE}". There's an entire universe of Linux-using systems out there that have operating constraints very different from your typical anonymous cloud server machine.
Also are shared libraries all that used in Android? The part of the system that is written in native code is very small, it probably doesn't make all that sense to use shared libraries for that.
The problems of updating applications individually because they're statically compiled are insignificant compared to the problems that come from having to do everything within containers.
Bundling dependencies aren't the only reason containers are used though. We still want to sandbox everything, and those sandboxes need to provide dependencies.
At least with dynamically linked you can decouple and share things like "runtimes" in the flatpak vernacular, which does have some advantages like dependency security updates.
System libs can be problematic because the OS's API may change between versions or different flavors of the OS.
For general apps using third-party libraries, nowadays static libraries is the way to go, as I have said months ago [0].
I kinda like how they open up opportunities for fixing or tweaking the behavior of broken proprietary binary-only garbage without requiring you to RE and patch a massive binary blob. Of course, in a better world, that garbage wouldn't exist and we'd fix the source instead.
If so, which downsides are those?
I can't tell if they exist, and it seems to me that this is a flexible solution that gives the power of choice to the consumer of the library.
> Disk is cheap
I certainly don't want each installed desktop app to have a copy of base gnome/kde runtime and everything down to libc. And the implication is even the graphics would be duplicated, for example the Adwaita icon pack is huge. So if I have a full set of gnome applications (say 50) would I have 50 copies of Adwaita icon set? Suddenly disk space isn't cheap. Shared libs are good and we could do better than flatpaks and containers and static linking.
And just because shared libs are PITA it's not just because of their nature, it's the lack of tooling from supposedly modern languages, lack of guidance, lack of care for versioning and API stability, and distro agnostic convention. Each of these problems can be solved by not sweeping them under the rug.
>Yes, it can save on disk use, but unless it's some very core library used by a lot of things (ie particularly things like GUI libraries like gnome or Qt or similar), the disk savings are often not all that big - and disk is cheap
He's literally making the point you're arguing. He says, core libraries should be shared.
C is the default choice because of its ABI, and the tooling around it is made for using shared libraries.
What can you say about modern languages? Each of the languages are designed to work in a silo and not much cooperation for let's say, system plumbing. Their own package manager makes it easy to pull in code, and only make code for that language only. You can't just create a package that works as a shared library for other programs without mushing around C FFI. They make it hard by default, which creates resistance in developers to make a piece of code usable by others other than their own language. This trend is pretty alarming, especially when hidden and manipulative language fanboyism is showing its ugly head everywhere.
No. Icons is easier to load with regular open/read, then with dlopen. But in any case they go into separate files, they are not in a binary.
Dynamic loading might be used to load data into the process, but it will be a very strange way to do it.
> And just because shared libs are PITA it's not just because of their nature, it's the lack of tooling from supposedly modern languages, ...
It is more complex than this. Dynamic linkage is limited in its ways. It couldn't do a type parametrization, for example. All it could do is to fill gaps in a code with function addresses. But it is not enough. Far not enough. For instance, you wouldn't like to dynamically link a c++ vector, because it is meant to be inlined and highly optimized then. Dynamic linker cannot inline or optimize.
So you are forced to use a lot of inlining at the stage of the static linking of the application binary, but then you get a problem of binary incompatibility between an app and a lib coming from lib being rebuilt with different optimizations.
So I'd say, that modern linux distributions should adapt to modern languages (like c++, lol), not vice versa.
> Each of these problems can be solved by not sweeping them under the rug.
For what end? What we possibly might gain, from solving these problems? Dynamic linkage is a runtime cost. Why should we prefer runtime costs to compile-time ones?
That of course will not help with RAM but should, reduce storage requirements quite a bit.
In general I agree with Linus, an application should not be separated in to multiple parts unless it serves the user.
The exception that everyone could get onboard with would be a common API that all programs use. So, for instance, if everyone could agree to use GTK instead of Qt, then GTK could just be shipped as standard on all Linux distros and there would be no problem of bloat - all programs need GTK.
But that goes against the Linux philosophy of modularity & configurability, so that will never happen either.
So you need to strike a happy medium. Make an attempt to use a shared library, but if your users don't have it, download it live.
Not Linus, but a lot of young dev and newcomers are discovering static libs building with GO and co, and think that it is a revolution and that it solves all their problems.
But, they are probably forgetting or not knowing the world of static libs before dynamic one became a thing.
A big part of the revolution of linux success in servers and all come from the magic that brought a full system built on shared dynamic libraries.
Do anyone remember having to install a 1Gb software to setup a hp printer on windows?
The full afternoon or day needed to wait for the installation of visual studio?
The ms office installation needing multiple gigabytes when openoffice could be downloaded and installed for a hundreds megabytes at most?
First of all, the dynamic linker is slow and not optimized for program startup times. [ https://lore.kernel.org/lkml/4b77489c05f6459ead7233c3fb69f61... ]
And second, compiling clang with Profile Guided Optimization makes a big difference. [ https://lore.kernel.org/lkml/YItU3YrFi8REwkRA@archlinux-ax16... ]
This does not really work in Linux.
Meanwhile, dependency hell on Linux distributions keeps people stuck on outdated versions of software, because some piece somewhere is either too old or too new.
do you think this is an accurate characterization of what happened
In addition, with regards to saved disk space and central patching, Docker negates a lot of those benefits. Basically instead of static linking, you end up shipping a whole container with a bunch of shared libraries
I'm a Fedora packager and I'm not sure what he's talking about here. If a single package uses some code then there's no requirement to make that code into a shared library.
> Packages including libraries SHOULD exclude static libs as far as possible
> In general, packagers SHOULD NOT ship static libraries.
[0] https://docs.fedoraproject.org/en-US/packaging-guidelines/#p...
[1] https://stackoverflow.com/questions/3430400/linux-static-lin...
Every docker has its own openssl version. Which means every openssl cve means updating every docker image. Awful.
It used to be you'd target RHEL X and be done for 10 years.
"Yes, it can save on disk use, but unless it's some very core library used by a lot of things (ie particularly things like GUI libraries like gnome or Qt or similar), the disk savings are often not all that big - and disk is cheap."
How much savings? Has this ever been measured? Just curious.
Heck, the heavy usage of Docker or similar in industry really signals that developers are more than happy to tradeoff disk usage, often a lot of it comparatively speaking, for reliability.
Regular string ops can vary in speed by 2-5x based on which old glibc you have.
what does ram usage look like if you build gnome, qt, x, wayland apps statically? seems like you might end up with weird stuff like huge rss for things like calculators and desktop tools. moreover, doesn't it just push compatibility problems to the network protocol/ipc level anyhow? i suppose maybe that's more stable... for now?
His way of saying "what you're referring to as Linux is actually GNU/Linux".
On Arch, users pick their display manager and windows system, and many Qt applications or GTK applications may share multiple libraries and I can see the benefit of smaller packages and being able to update those libraries separately.
Go already has several other deal-breakers for systems programmers (garbage collected everything, runtime-provided greenthreading, etc). Rust is a lot closer to what Linus would approve for in-kernel development, but it still has a few problems, such as failed allocations panicking the system instead of returning an error that can be handled normally.
Can we get a app/service to read these in some sensible way? That would be awesome.
First off, dynamic linking does not seem to make updating software impossible. There are several rolling release Linux distros (Archlinux, Debian-unstable, Void, Gentoo, etc.) and basically everything works while releasing upstream updates within 2-4 weeks typically. Most are maintained by volunteers. The number of core packages that break due to shared libraries is single digits in my experience of Archlinux over the last 5-8 years.
Sure some things are shipped as pre-compiled packages which just come with their own library directories so they can load the versions they need.
I will say that I have encountered software that was library hell. This software tended to be academic in nature and use really poor build processes like having dozens of make files while being developed on ancient versions of Ubuntu. The software was not vaporware or abandoned it was just poorly developed. Even if static linking would help - the developers choose not to use it out of ignorance.
Other software that has issues with libraries tend to be very large applications where the overhead of shipping an alternative LD_LIBRARY_PATH or 'Docker' container is minimal and has other benefits as well.
I cannot imagine how many projects will actually keep their libraries up to date. Electron apps are already often using old Electron versions causing issues where the Electron version won't support Wayland as well as the browser version. I have stopped using the desktop versions of apps like Microsoft Teams and Slack because of this issue. If large companies can't be bothered to update their flagship desktop applications libraries - how can we expect small, open source projects to do better? They no doubt use community feedback from distro maintainers & users to maintain compatibility with upstream libraries.
Plus if static linking is used to an extreme amount then their will be N-~1 more versions to 'maintain'. Any popular library/dependency will become a defacto supported version; just look how long it took apps to move to Python 3. Heck, the 'chromium' package seems to still require Python 2 as a build dependency. Even 'Cellebrite', as evil as the program is, was hacked in part because it shipped with out of date libraries and it is probably has a multi-million dollar price tag.
TLDR: Developers of all levels suck at keeping dependencies updated. The distro system & build process provide s a much needed barrier between application development and sanity. It's basically the final code review for the Linux & open source ecosystem. That said static linking certainly has its place but also has its own set of issues.
I see it as a huge weakness on the GNU/Linux world because it's so widespread.
In Windows, "DLL hell" used to be a thing. It's mostly solved by (among other things) Microsoft forbidding installers from overwriting system DLLs. Also today most apps bundle the DLLs along with the apps and don't really share them.
In MacOS, brew the popular package manager recently took upon using shared libs for everything. While it mostly works it can cause some version incompatibilities, you tend to accumulate garbage in your disk after uninstalling stuff and now you have to routinely update 3x more packages...
/s
Abstraction is good. Surely anyone can see that having one copy of code is superior - technically, and from a standpoint of elegance - than duplicating it once per program. A huge amount of effort has gone into making this infrastructure work. This isn't just about disk space, or memory, or the ease of patching - it's about composability. I like that I can override behavior with LD_PRELOAD. If we think of our systems as a collection of unrelated, opaque blobs, then static linking looks seductively simple; but if we think of them as a network of interacting code that transcends application boundaries - vivisected and accessible, like a Lisp machine - then static linking is a repulsive act of violence against deeply held aesthetic principles.
If dynamic linking doesn't work well - if we've messed it up - that's a sign we should try harder, not give up.
Not for Linux, though, because that's not the kind of operating system Linux is.
Torvalds: [hot take that almost no one else agrees with]
Considering the popularity of vendoring and/or static linking dependencies, I'm not sure where you're getting the "almost no one else agrees with" from.
glibc's hostility towards static linking is arguably a substantial part of why musl has attracted so much interest. Static linking support is explicitly stated as one of musl's core tenets [0].
One can also argue the popularity of containers has in part been due to glibc's dynamic linking requirements, and containers have become the bloated "modern" version of static linked binaries, bundling all the dependencies without eliding all the dynamic linking overhead.
Torvalds is definitely not alone in this particular opinion.
Vendoring is hugely popular for projects in year zero, because it makes it easier to ship. Until they learn that packaging specific versions of all your dependencies with your project implies taking on responsibility for backporting all security updates to that version of every dependency when upstream is only actively maintaining the latest version. Or you have a project riddled with known published security vulnerabilities, which is what most of those turn into.
Anothre reason is that glibc is equally bad for dynamic linking. Often a binary compiled on recent glibc doesn't work with older glibc. It is certainly possible but quite challenging to compile portable executables on Linux.
These people are propagandizing the bogus security argument that static linking is bad practice.
Take the red pill: statically linked executables are actually good practice.
Aren't most of the people working on distros doing it for free.
Here, I fixed it for you!