GNU Guix 0.5 released
lists.gnu.org
lists.gnu.org
If you like this approach to package management, you might want to consider trying NixOS[2], which is a distro that uses Nix as its primary package manager. NixOS has a disproportionately high uptake in the Haskell community, so it has a very good selection of Haskell packages--if you're playing around with or actively using Haskell, that could be another bonus for NixOS.
Personally, I think the idea is very cool. I will probably at least try using NixOS for my next laptop--I generally switch distros every time I get a new computer.
[1]: http://nixos.org/nix/ [2]: http://nixos.org/
One problem I ran into as a ruby guy was building gems that needed to link against a C lib. Part of the beauty of the Nix approach is that it isolates everything. The "proper" solution I guess would be to package gems as Nix packages, but it seems like there probably should be some way to let a utility know the proper build flags for a given lib.
For each language they have an adapter that exports the language-specific package manager to nix expressions. cabal, perl, python and ruby are all supported (maybe more). In the case of ruby there is `nix` ruby gem that you can use to generate a .nix expression. Ideally you could amend it to say that nokogiri depends on libxml2 and such. Then your project could have a deps.nix file that you can install. `nix-env -f deps.nix -i`
I'm also exploring different ways to work with my projects. For example creating a custom profile per project that's derived from a project-specific derivation.
I'd rather we collectively agree on a protocol for installing, updating, querying packages etc, such that each language-specific package manager can talk to others through a daemon and install foreign packages that are required for the projects. It would require a fair amount of work to update each package manager to talk it (optionally), and agree on the semantics and policies of the protocol, but I think the benefits could be huge.
Using Nix/Guix as the basis for such protocol would probably be ideal due to the immutable nature of packages and ability to install side-by-side versions, and that you can specify an identity of a dependency, rather than rely on fuzzy matching of name and version.
Honestly, the first one (of Nix vs. Guix) that can replace dpkg on my (non-NixOS) system is welcome! For now, having this "unstable" in the default nix channel (http://nixos.org/channels/nixpkgs-unstable) is scary.
Also, the selection of packages in Guix is way too small. 600 advertized, vs. nearly 10000 for nix. Is Firefox even in? (no. Icecat is, whatever that is. Don't ask if Chrome is in.)
Another thing: Guix seems to default to per-user package installation, unlike Nix. http://xkcd.com/1200/ comes to mind. In the case of Nix, "you can steal my cookies, but you cannot list Nix packages I can install!"
This is a 0.5 release because there's a lot left to do. There are tons of packages left to write in order to have a good selection of free software. We would love some help.
>Is Firefox even in? (no. Icecat is, whatever that is. Don't ask if Chrome is in.)
GNU Icecat is a fork of Firefox with the nonfree bits removed. Chrome is proprietary so it will absolutely not be part of the GNU distribution. Chromium has licensing issues (and is a nightmare to package, btw: https://twitter.com/spotrh/status/410479218791706624). I don't know of any fully free distro that packages it so I don't think it will be available in Guix.
However, if you do want to use proprietary software, you could always pull packages from other sources that package such things.
Why would I use one and not the other?
I recommend checking out some of the package definitions in the gnu/packages directory to see how elegant the DSL is. Here's a module that I wrote: http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/...
If you happen to be interested in monads in Scheme, be sure to check out guix/monads.scm. It's a good read: http://git.savannah.gnu.org/cgit/guix.git/tree/guix/monads.s...
Please try out the QEMU image for an early look at Guix's GNU system. If you like what you see, we could use some help packaging popular free software programs. Oh, and join the mailing list and come lurk in #guix on freenode.
Though personally, the licensing section leads a real sour taste in my mouth. Nix's licensing system is much simpler, much less politicized. Simply link to the license directly, leave the commentary elsewhere. Linking to the FSF's reasonings for every license just makes me much less inclined to experiment with it.
>The result of package build functions is cached in the file system, in a special directory called the store (see The Store). Each package is installed in a directory of its own, in the store—by default under /nix/store. The directory name contains a hash of all the inputs used to build that package; thus, changing an input yields a different directory name.
>This approach is the foundation of Guix’s salient features: support for transactional package upgrade and rollback, per-user installation, and garbage collection of packages (see Features).
To break that down, first: Can you get a determinisitc build from using a functional package manager?
Second, is a functional package manager a better way to get deterministic builds than whatever else is used (and what that is, I have no clue)?
The end result is that the final fingerprint of a build artifact is extremely sensitive to absolutely every choice made during the build and it's very hard for the artifact to depend on anything that hasn't been deliberately chosen in the build scripts. Together these facts help to ensure a kind of global, repeatable build determinism pretty much unheard of outside of Nix.
[0] The project Guix is forking
But the name is horrible. Guix makes this sound like something GUI-related. A GNU implementation of X? I don't know. The name is wildly misleading.
They really should consider doing something about that, because naming conventions tend to matter more than you think they do.
Time to fire up my yeeloong
First of all, requiring the directory you install shit to to include a full cryptographic hash in its name? Do you even comprehend how completely useless the file path is now to a human? Sure you can have a directory of symlinks to make it readable again, but as we should know by now, plenty of applications bork on trees of symlinks (and you can't hardlink a directory). Plus your garbage collector depends on hardlinks (or a whole lot of tree indexing). This could have been managed in so many ways to make it a lot more user friendly.
"Complete dependencies"? You mean MANDATORY dependencies? This is so fucking annoying. Now I can't install an entire OS because one package with a specific SHA is gone. So even though I could rebuild it from scratch and it could be functionally identical, with 1 bit different the SHA is hosed and now everything is useless. Not to mention even if I had it, I have to ship every single dependency to anyone any time I want to give them that package/app/etc. Fantastic. And 'scanning binaries for hashes of directories'? This has got to be the laziest, dumbest solution to a complex problem i've seen supported by GNU since...... Wait, nevermind. They've probably supported dumber things.
MULTI-USERS?!? YOU MEAN A NORMAL USER CAN INSTALL A PACKAGE? GET OUT. NO PACKAGE MANAGER CAN DO THAT EXCEPT.... All of them?
Atomic rollbacks/upgrades sounds nice, until you realize a program might be running other programs, which might trigger your runtime dependency checking, which might screw everything up.
"Garbage collector" ? Oh. If it has hardlinks, you don't delete it. Gotcha.
As far as "functional" operations go, it's been pretty well proven in production environments that you don't actually have to completely recreate a binary exactly; just approximately. Functional operations management gives techies a hard-on, but means diddly in terms of real-world problem solving. We have managed away the implicated problems years ago.
WHY ARE WE STILL INSTALLING FROM SOURCE?! Did Gentoo teach you all nothing!?!
Service deployment? You mean service configuration management and deployment? Sigh, here we go again. Let's combine three different things into one and pretend it's a viable solution for general operational challenges.
Everything else seems handy.
If we were using a relational database and we had a table for packages with "primary key","name","version", it's not sufficient to query name and version to get a package, because it's not guaranteed to be unique. For example, you might have the same version, but built using a different configuration, or having a bug fix. You might have revision 127 and 128 in there, but they're the same "version" of the software (I'm not aware of many packages which put source control revision numbers into their release version numbers.) The only way the developer can tell others precisely what he means is to specify an identity, and the rest of the information can be queried.
The hash serves another useful purpose anyway - the ability to verify a package. At the moment Nix hashes the build instructions of a package, so it's not necessary to create an exact binary, although that would be desirable when we can do so reliably. We could eliminate the need to trust package maintainers we don't know this way, because cryptographic hashes are unlikely to lie. Nobody inspects every binary package released publicly to confirm the packager hasn't inserted some malware, but it would be desirable to have the package checked if the computer can automate it. Oh, and you don't need to install from source with Nix. You can download binary derivations, but until we have exact reproducible builds, you do so at your own risk.
Configuration management and package management are not disjoint ideas, and nor do other systems treat them that way. In puppet for example, you can hook into another package manager to install new packages with a fuzzy match, then configure them. Nix configuration is just the same concept for the Nix package manager - where you can actually specify the identity of some configuration you depend on, and in turn give your configuration an identity that others can depend on.
Of course Nix is not a silver bullet, you trade away some of the convenience of other systems in exchange for having reliable distribution of software exactly as the developer/maintainer meant for it to be, and without having to cross your fingers and hope it builds OK.
All [modern] packages have unique names without the need for a completely unique identifier embedded in the file name. The file name is guaranteed to be unique by the distribution maintainer of said package. Usually for 3rd-party packages a postfix of a revision number and the distribution from whence it came is what makes it unique for that distributor. This ends up not only giving you insight as from whence the package came in the name, but which revision it is (if listing them sequentially, for example).
But it isn't package names that need to be unique, it's file paths delivered by the packages, and that certainly is a useful thing that's good to solve with a package manager. But you still need to be able to discern with your eyes what is what! You can't easily do that by comparing a gigantic SHA hash between two file paths, but you could do that if the difference between each other was very minor, but indicated what each path actually was related to (in terms of version, arch, revision, order of installation, or any other number of criteria).
I get it - you're trying to support a completely universal, will-never-conflict file path. But we don't need that!!! It's simple to maintain a small, mostly-collision-free postfix to a package name that is both human readable and fairly unique.
Another way to point out the ridiculousness of the hash-in-file-path system is in your first paragraph:
> It's not like relational data stores use human readable primary key fields is it?
File paths are not relational data stores. File paths are a human interface. A program can store and retrieve data in any way it chooses, and in much more efficient forms than filesystem paths that use "special character" separators, are subject to various environment changes, language specifics, platform differences, and name length limitations. We use file paths because they're the easiest way for humans to organize data files on the disk. Hell, their inherent crappiness is part of why many applications have to be built to "scale" how they manage files to get around the inherent problems of human-readable file paths. Sticking a grotesque amount of non-human-readable metadata in there is like trying to shove a calculator into a pencil.
> The hash serves another useful purpose anyway - the ability to verify a package.
What the hell is this doing in the file name, though? Packages already include hashes of themselves and their files. You can verify any of them with the record they create in the package management database, or in the original package.
I don't get what your point is about people "lying" about the binary package or inserting malware. We've had public key-signed packages for decades. This is a non-issue.
Config management is completely different from package management. I know there are systems out there that try to build in hooks so you can use one from the other, and i'm not saying it's completely useless. But in general it's crap to try to do the functions of one tool in another tool. They need to exist independently for obvious management reasons. The more you tie them together, the more of a maintenance headache you are left with.
Lastly, i'll just say that my favorite Linux distribution (in terms of stability, reliability, etc) is Slackware. It basically defies all of the reasons Nix seems to exist for, yet it works beautifully in spite of it. It turns out that you don't have to wrestle and fight with your packages to make a system work well.
I don't disagree that they could do better, by perhaps using a postfix of some smaller signature of the package, but I just think the argument is irrelevant, because they're were not intended to be read by humans - they are handled by the Nix software. I've not needed to manually browse into /nix/store to find something yet.
File paths are primary keys in a relational data store which is the filesystem. The filesystem is crafted for quite a specific purpose, so is not as general as say, sqlite. Nix has relational data and uses the filesystem because it needs to be compatible with the rest of the system. We could in theory use sqlite to store everything we need for nix, then mount the database with some hypothetical FUSE filesystem to reveal it to the common namespace in some pretty way. That just requires additional effort for no real gain.
Public-key signing does absolutely nothing to verify that a package was built from an unmodified source using an unmodified build process. Signing merely proves who the package came from (that nobody has modified it since the packager built it). It does not identify whether or not they have ill intent. We should learn from recent NSA revelations and cases like Lavabit, that they may be forced to insert malicious code against their own volition. We can't inspect binaries and make sure they match up to the source code, but if we had reproducible builds, then N people could build the same software, resulting in the same binaries, and we could have a web-of-trust model, where you only install a binary package if there's a consensus on it's hash, and have a pretty high confidence rate that it's safe (A malicous party would need to produce a hash collision). Trusting N > 1 randomly selected people is probably better than trusting 1.
Part of the reason for having configuration management on top of Nix is much the same reason as the package manager itself. The aim is to remove "hidden knowledge" that the packager or administrator used to build or configure some service, so that everything required to reproduce it exactly is well defined in a declarative programming language. Existing CM services like Puppet are unsuitable for the task because they make many assumptions (such as how other package managers work, or that there's only one derivation of a package on the system), that are broken with Nix. A new CM service which removes those assumptions was needed, and it happened to be built on top of Nix because it can assign identity to a configuration.
There model has other uses. Hydra for example, is a continuous build system which uses nixpkgs. There's potential for replacement or improvement of existing build tools like make (which have plenty of flaws). pkg-config isn't necessary for example, when we're dealing with dependencies which are specific by design. It's not that it's "combining many different things into one", but rather, rebuilding a bunch of services upon different assumptions than what is already available.
It should be noted that you can use the package manager without any CM, but NixOS uses the CM ubiquitously to give a clear specification of the OS and how to reproduce it from scratch.
For me, Nix is a step towards removing the distinction between developer and user. The future I see is where there is zero friction to go from using an app to hacking on it, because you have the source code readily available, plus a completely automated build process you probably don't need to mess with. There's no hidden knowledge. I cannot count the amount of time I've wasted in the past trying to build software and dependencies of software because they're not well defined or documented. Quite often a day has passed before I can actually start modifying the code.
You don't need a declarative programming language to for your configuration management or your package management. It just makes for a maintenance headache. And using a dedicated CM tool like Puppet is a better idea because it makes the software more portable and flexible.
When you deploy your application, your package manager has to execute certain things in order to set it up for the first time. A CM can handle this, too. And once it's installed, you may need to change the configuration in a unified way. A CM handles this. At uninstall, your package manager cleans up after the package's setup, which a CM can also handle. By bundling these two, the application now only supports one system (the Nix system). If the software's configuration is completely handled by an independent CM tool like Puppet, it will be supported on any platform Puppet resides on, and because it's not tied to a single package manager you can install the software on any system.
I think the only way you're going to win the development war is to provide a portable, compatible, simple, maintainable, easy to use tool that everyone will want because it solves all of their problems without creating new ones. It can't be a programming language because most people can't program. It can't be complicated or overly powerful because most people will shoot themselves in the foot if you give them ammunition. And it has to work everywhere, on anything. Good luck!