Haiku package management
markround.com
markround.com
libsolv, created by Michael Schoeder at SUSE, has two innovations:
- a memory optimized format to represent packages and its dependencies, using hashed string pools - a SAT solver able to operate directly in this representation, battle tested in this particular scenario of complex upgrades, including hundred of testcases.
Haiku uses mainly the solver only, as far as I saw in the source.
https://github.com/openSUSE/libsolv/blob/347200107f0f17a3697...
Not sure what my point is, I was just confused - I know SUSE/Fedora people are close
I’m also super excited to see Haiku mature. Especially once it passes that breakaway threshold where it’s not primarily being run as a VM and the third-party ecosystem starts getting traction.
>Just because they never got wide adoption doesn’t mean they were wrong
This is actually something that I was thinking about recently, applies to a lot of topics not just OSes
Check out (as you mentioned) NixOS or Fedora Silverblue
I have a question about this, please forgive my ignorance. I see this statement repeated a lot about NixOS and I've kind of had some "success" in the comments section being "critical" / "asking hard questions" about NixOS.
If I have a build script that pulls + compiles + validates with hashes/checksums say, Linux kernel, some userspace tools, init system, etc. for a boring ol' distribution like Alpine or Debian, why isn't that classified as "reproducible" the same way NixOS is?
Nix, on the other hand, is reproducible by default. If you `nix build <nixpkgs/gitsha#pkgname>` you are automatically getting a local build that takes place in the same hermetic sandbox as the build you pulled from cache.nixos.org, and will be identical or very close to it. (Contrary to popular belief, not all of nixpkgs is reproducible— many packages require patching for various sources of nondeterminism, and this is an ongoing effort: https://github.com/NixOS/nixpkgs/labels/6.topic%3A%20reprodu...)
Other previously nondeterministic build issues have some similar opt-in workarounds. Have a look around the Reproducable Builds project website for more info/examples.
https://reproducible-builds.org/docs/source-date-epoch/
https://gcc.gnu.org/onlinedocs/cpp/Environment-Variables.htm...
Older package managers use a "name-version" scheme to try and avoid package conflicts, but in practice there can be different binaries which have the same "name-version" identifier and they're not always compatible.
The way Nix solves the issue is to store everything in a content-addressible store `/nix/store`, where multiple concurrent versions can be installed without naming conflicts (the directories are named with the content hash). When an app is compiled, it is linked to specific shared libraries in the nix store which will always be present on the system. The `/usr` directory then, is just a collection of symlinks to the actual content in the nix store (most recent versions), for compatibility.
Because derived packages are reproducible, they will always have the same content hash on every system on which they are deployed. As a developer then, you can target very specific shared libraries when building your software, and you know that when you deploy them, users are going to have the exact same dependencies you had when you built it.
It might be! Nix uses that very same strategy for enforcing reproducible outcomes when dealing with non-reproducible operations, like fetching things from the great mutating mass of the internet.
I would say that Nix differs from that approach mainly in the way it divides up the problem. So-called binary package managers like you use on Alpine and Debian perform runtime dependency resolution against a mutating global namespace that usually lives on the internet— the binary repos themselves. Generally, those repos include only one copy of each package. So to halt the forward march of all those versions, you have to mirror each repo of binary artifacts you're interested in using.
With Nix, outcomes also depend on the state of the repo, but the repo is just a collection of build recipes which are the mapped onto their outputs in a single, huge binary cache. Every new build gets added to the same binary cache, for every Nixpkgs/NixOS release. It just grows and grows. And of course, Nix builds work even without enabling the binary cache at all.
So to get the same kind of reproducibility out of many other package managers, you generally have to set up additional infrastructure.
But if you do everything right, you can certainly have a fully reproducible OS image based on Alpine or whatever else.
1. Implicit dependencies that are needed to build the software but not managed by the package manager or build system itself. For example, if your language package manager depends on a C library that it can't build itself. Or Linux distribution package managers that implicitly assume the presence of a "base system".
2. SAT solvers for version selection. Using one of these usually introduces an implicit dependency on the clock time, because your dependencies will resolve to different versions at different times.
The reason your script wouldn't classify as reproducible is simply that you would probably fail to fully specify your dependencies. It's harder than you'd expect once you start moving up the software stack to complicated high-level applications.
Yeah, this is similar in spirit to what Flatpak and Snap do. The latter even uses SquashFS images it mounts for every package.
I have a feeling, though, that this is more similar to Distri¹, where the package manager is a bit more traditional. That is to say, packages are fine-grained (not packaging large runtimes into single chunks, like Flatpak and Snap) and there's not as much special tooling as Flatpak has for redirecting inputs and outputs, sandboxing, etc.
Distri also eliminates all hooks and triggers from packages, leading to super fast installs, which is pretty cool.
> Or I guess, I’m not an expert, that’s what NixOS is trying to solve?
From the article, three key features highlighted are:
> - Immutable system directories
> - Rollback to previous states
> - User-managed packages separated from system packages
But spiritually I'd say the basic idea (avoid dependency hell through lightweight package isolation) is similar and so are the results, at least in terms of capabilities.
PS: If you're interested in another example, RedoxOS' package manager also has a very similar design to Haiku's and Distri's, IIRC.
—
2: Because NixOS allows you to switch to a new system without booting it (i.e., it lets you upgrade without rebooting), it actually has two symlinks here: booted-system and current-system. The current-system symlink rotates as soon as you switch configurations, but booted-system gets set only on a new boot.
Another thing BeOS did well (and it's still ahead) was the filesystem. The mail program was implemented directly in the filesystem: https://arstechnica.com/information-technology/2018/07/the-b...
At one point in history, there was a chance that BeOS would be adopted as the Mac OS. I wonder how that fork in the road would look like.
If Nix's solution were more elegant in this sense, you wouldn't be able to use it on macOS because macOS lacks the required sandboxing features.
I love playing from time to time with it. If there would be ARM64 support (I wish I could find time to contribute...), combined with the progress of Asahi Linux, it would be fun (and a pun to history) to run Haiku on Apple M1s!
Well worth a read and fascinating (to me, anyway!) to see new ideas and approaches from the Alt-OS scene.
[1]=https://lobste.rs/s/j9decl/haiku_package_management
[2]=https://lobste.rs/s/j9decl/haiku_package_management#c_owqi3t
Definitely going to look into Haiku more myself :)
Two founts of hope
For computing's future
"SerenityOS is a love letter to '90s user interfaces with a custom Unix-like core. It flatters with sincerity by stealing beautiful ideas from various other systems.
Roughly speaking, the goal is a marriage between the aesthetic of late-1990s productivity software and the power-user accessibility of late-2000s *nix."
I think in terms of paradigm, Serenity has a stronger ideology than RedoxOS. It brings BeOS and Mac concepts like a consistent and fully integrated monorepo system and development kits, and a very strong and defined user interface paradigm.
From that perspective I find Redox much less of a paradigm for the end-user, if the main feature is the language the developers used. Orbital does not look remarkable.
The level of productivity of Andreas himself, his videos and the community he managed to built, including growing a few individuals of to major contributors to the point they created a whole browser stack is just outstanding as an open-source case study. Also to me, much more interesting than a programming language used to built it.
Haiku is written in the same language as Serenity, btw.
I'd add to that, Andreas is creating his own memory-safe compiled language to build components for the OS. Jakt has already been used to build things, including its own compiler.
Also I think what makes both HaikuOS and Serenity compelling is the common vision and clear direction: Aiming to recreate then improve BeOS has given HaikuOS project a clear steer ahead, while Andreas' clear guidelines and monorepo has done the same for SerenityOS.
Glad to see it still thriving and making the news every once in a while.
(I learnt about Haiku’s similar approach after building distri.)
https://michael.stapelberg.ch/posts/2019-08-17-introducing-d...
Other than that it seems solid as long as your hardware is well supported. Think I will commit my laptop to it this weekend and see how long it lasts.
I would trust the NTFS driver pretty implicitly at this point both for read/write; it's based on NTFS-3G, and following the rewrite I did of the Haiku-specific parts of it in late 2021, appears to be very solid in testing.
The EXT2/3/4 driver is also very good, but there may be some issues remaining there. However I haven't heard reports of it destroying data.
The FAT driver, on the other hand, has some cases where it has been known to corrupt directories. It could probably use a rewrite; most of its code dates to the 1990s...
Those are the "big 3" of other filesystems, anyway. I don't think there's a status page for all of them.
ZFS and Btrfs both are very good these days?
Can linux mount the Haiku version of BFS? Seem to recall that this was an issue the first few times I tried Haiku and it was buggy and unreliable but don't quite remember or recall trying it since. Searches are not pulling up anything useful. If not is this something Haiku is planing to work on or are you just going to let linux take care of that when ever they get around too it?
A 3 year old ticket with only one reproduction from 2 years ago, and it appears to have been fixed in the interim, and now the ticket has been closed by the main developer of the ext2/3/4 driver. I wouldn't be too worried about that.
> I don't think I will be plugging in my primary backup drive just yet
That's probably a good idea anyway.
> but at least it can now deal with journaling on ext4, progress.
I think it's supported journaling for many years, I don't believe that to be a new feature.
> ZFS and Btrfs both are very good these days?
We don't support ZFS at all, and btrfs support is read-only (there's experimental write support but it's disabled in default builds still, I think.)
> Can linux mount the Haiku version of BFS?
Yes, read-only with the upstream Linux driver. There is a way to get read-write support with Haiku's driver under FUSE, however it's not very performant to say the least.