HNHacker News
TopNewBestAskShowJobs

pg83

139 karma · joined March 25, 2023

dev, math, c++, go, python, build systems https://stal-ix.github.io/ anton@samokhvalov.xyz
submissionscomments
pg83··on Solo – a .so loader for static Linux binaries
The README.md link describes cosmopolitan's approach, among other things, and it's worse than mine.
pg83··on Solo – a .so loader for static Linux binaries
> i'm not sure how much people realize that the modern "graphics driver" is actually just "the kernel multiplexes userspace messages to/from the GPU and we've taught mesa to understand each family of GPU you'd ever care to support."

In the case of OpenGL, it has a fairly sophisticated state tracker, which you can carry around with you. For example, I can statically compile ANGLE this way.

But even OpenGL/Vulkan have a shader compiler that's unique to the hardware (it's expensive to carry it around with every piece of hardware in the universe).

And I'm not even talking about libdrm, which is quite tightly tied to the kernel version.

I know exactly what I'm talking about, because I can also statically compile Mesa into my application.

pg83··on Solo – a .so loader for static Linux binaries
Or may be not.
pg83··on Solo – a .so loader for static Linux binaries
Any software WILL break on future distributions, sooner or later.
pg83··on Solo – a .so loader for static Linux binaries
These decisions, and these studies, were made a VERY long time ago. It's completely unclear why the decisions made then are relevant now, and why they can't be challenged.
pg83··on Solo – a .so loader for static Linux binaries
I disagree that this is a niche problem.

Yes, on the one hand, each specific distribution doesn't have this problem because it can pick up everything it needs.

But for us, independent developers of small programs, the problem is truly stark: we can't afford to build our programs for every distribution. And we can't afford to waste time navigating all the idiosyncrasies of various package repositories, both technical and political (not all repositories allow easy access).

And we can't count on someone else packaging our work until we become incredibly popular.

pg83··on Solo – a .so loader for static Linux binaries
It's so good that Linus thinks differently!
pg83··on Solo – a .so loader for static Linux binaries
Thanks a lot, guys!
pg83··on Solo – a .so loader for static Linux binaries
> A change in paradigm is necessary. Freestanding C, not hosted C. This completely gets rid of the libc and is a surprisingly clean language. Linux only, because it's the only kernel with a stable binary interface. Every other OS forces a C runtime.

Great choice for small programs, but what if I want hardware accelerated 3d?

pg83··on Solo – a .so loader for static Linux binaries
Of course, I remember those times very well.

And I also remember very well that dynamic linking appeared ONLY because we were catastrophically short on memory; everything else was added much later.

Now we have plenty of memory, and we can very well return to our blessed roots!

pg83··on Solo – a .so loader for static Linux binaries
This means the user will update the binary with my program and continue using it happily.

I'm not offering a silver bullet, but the approach I've implemented is much better than what the industry currently offers.

pg83··on Solo – a .so loader for static Linux binaries
> One advice I'd give you is to _really_ take care regarding SysV/ELF ABI conventions, there's tons of undocumented stuff in there and it's really easy to mess something up or cause a security defect (see AT_SECURE)

Yes, it's not trivial, but I hope that over time everything will settle down.

> if I understood your implementation correctly you're hooking this into an already running musl which could cause backwards compatibility issues if musl changes under you

No, I hook this to musl, which is statically linked into my binary, and I have complete control over it.

pg83··on Solo – a .so loader for static Linux binaries
https://news.ycombinator.com/item?id=49358538
pg83··on Solo – a .so loader for static Linux binaries
Windows handles this much better than any other OS. The API passes versions (== structure sizes on the calling side), and WinAPI can handle that.

As for older Windows systems not being able to load new DLLs, they can; the format hasn't changed in a very long time. I've had experience with installing some DLLs on Windows NT 3.51 and running a modern Firefox, which is about 20 years behind the times.

pg83··on Solo – a .so loader for static Linux binaries
> And that's the correct approach.

Who said that? This approach has many problems that have already been discussed here, not to mention the fact that it leaves Alpine and Bionic-based systems out in the cold.

Let me remind you that Bionic is the most widespread libc in the Linux world, and Alpine is the most popular Docker layer.

The world doesn't end with glibc. And it doesn't begin with it.

pg83··on Solo – a .so loader for static Linux binaries
This approach only works for very simple cases.

In complex cases, it turns out that the old version of glibc also pulls in other libraries and the compiler, and you're stuck with a very ancient sysroot. You may often find that you can't compile new library versions in such a sysroot and link them statically.

So, it looks good on paper, but forget about the ravines.

pg83··on Solo – a .so loader for static Linux binaries
In this scenario, I'll have to choose which libc I want to run. These won't be portable Linux binaries in the true sense of the word; I'll have to leave Alpine out, and possibly Android, which I don't want.
pg83··on Solo – a .so loader for static Linux binaries
> Or even just write in your native language and then have Claude translate.

That's exactly what I did.

pg83··on Solo – a .so loader for static Linux binaries
Technically, you're right, it's a dynamic loader. Technically, it's pure dynamic loading.

If we look at the issue at its core, we're still a statically linked program in a hostile environment, forced to dynamically load device drivers from the system.

It's similar to Golang; on MacOS, it has to use libSystem, even though otherwise, these are the statically linked Go binaries we're used to and love.

Let me add a little more detail: if I use vdso with gettimeofday in a statically linked program on Linux, am I still a statically linked program, or not? :)

pg83··on Solo – a .so loader for static Linux binaries
> Do you belive the machine is taking offence? It can not.

Obviously, this is an offense to me.

> Telling me, a coder you never met or interacted with before, that I would introduce more bugs than The Product(tm) is pretty rude and prejudiced.

Don't exaggerate. I didn't say that you personally introduce more bugs (as you rightly pointed out, I don't know you and don't know how often you introduce bugs), I said that the average developer introduces more bugs than an SOTA LLM.

pg83··on Solo – a .so loader for static Linux binaries
Modern models, when properly managed with a human in the loop, write higher-quality code than humans and introduce significantly fewer bugs. Therefore, it's quite the opposite - you should expect fewer "dlopen-related crashes and/or intermittent bugs."
pg83··on Solo – a .so loader for static Linux binaries
> spew

Rude.

It only gets better from there - https://github.com/pg83/solo/blob/main/CONTRIBUTING.md!

pg83··on Solo – a .so loader for static Linux binaries
It depends on how we define the ABI. I see it as a set of client-visible invariants that they rely on. In my world, glibc changed the client's visible invariants, breaking the client. The client works on one glibc-based host, but not on another. What is this if not "a case of cross-distro glibc issues?"

Overall, both of our points of view on compatibility were discussed well in that thread; we probably shouldn't repeat ourselves. :)

pg83··on Solo – a .so loader for static Linux binaries
It's a pretty poor solution, to be honest.

1) Why should I limit myself to the available APIs?

2) Not just glibc. For example, if I build against the latest libstdc++, it will automatically support the more recent glibc. And pinning the old libstdc++ -well, that's just not a good idea.

pg83··on Solo – a .so loader for static Linux binaries
Well, ld.so already does this, and it's no big deal. The Python interpreter also does this when executing a .py script (code is code, whether it's machine-readable or human-readable).

In any case, we take testing very seriously—every glibc shim we've written is covered with tests, and we run our loader against 1000 of the most popular Debian packages. The project has 100% code coverage. Perhaps, if I have the time, I'll also do some fuzzing on this thing.

pg83··on Solo – a .so loader for static Linux binaries
In any case, it's open source. If you don't like something, even if the project seems generally useful, go ahead and fix it. The PR came in. I'm an engineer and I can write good code, but that doesn't mean I can write good README.mds!
pg83··on Solo – a .so loader for static Linux binaries
Rough
pg83··on Solo – a .so loader for static Linux binaries
The author of the README is me, the machine just wrote it. I am not a native speaker of English, my written English is simply terrible, no one wants to read the text that I wrote exactly :))
pg83··on Solo – a .so loader for static Linux binaries
Available in README - https://github.com/pg83/solo#how-this-differents-from-prior-...
pg83··on Solo – a .so loader for static Linux binaries
https://bugzilla.redhat.com/show_bug.cgi?id=638477 is the most "famous" example.

There are also much less well-known "little things" that regularly pop up here and there.

> If your CI uses an older glibc you should be fine AFAIK.

In any case, my binaries work not only under glibc, but also under Alpine, and (work in progress) under android/bionic.

Page 1 of 3Next →