And this is an awesome idea. Cryptographically hashing your install and keeping the install spec around for later is very powerful.
> Lock files, dependency resolution algorithms, etc. However, these are obsolete.
These are still very important (and hard) problems. Claiming that they’re obsolete ignores what they’re used for.
> The functional package management paradigm approaches the problem from a different angle which is able to precisely describe the full dependency graph for a piece of software, all the down to the bootstrap binaries (the C compiler's compiler, etc.).
I suppose this is fine from the perspective of a system package manager. But at the application development level, there are not very many users who want to specify all that.
If you want to play around with different software stacks in Guix and Nix, you have to actually write packages, which involves tweaking a lot of files. There aren’t so many people with the intestinal fortitude to get down to that level. I want a system where I can say “try running with a different version/build/configuration of this particular dependency”, and then the PM figures out what else needs to be done to accommodate that. Users don’t want to tweak package files a lot, and even less do they want to manage the profusion of version/configuration-specific package files that result from doing such experimentation.
We’ve developed Spack as kind of a compromise between these extremes. Spack [1] has builds parameterized by version, compiler, build variants, compiler flags (soon), etc. and it attempts to let the user experiment with a large combinatorial space of packages. You can do that with a command-line syntax (without editing package files), and you can specify tweaks down to the dependencies with a recursive syntax [2].
I support code teams at LLNL who want to experiment with many different compiler flags, dependency versions, etc., across Blue Gene/Q, Cray, Linux, GPU, and Xeon Phi machines, and none of them want to specify everything quite as rigorously as Nix/Guix demand. What we want is really good build parameterization and a concise way to specify it. I don’t see many Guix/Nix builds written that way — they’re all tied to specific versions and you’d need a new package file to support a new version.
> functional package managers overcome the problems that plague traditional package managers, allowing for unprivileged package management, transactional upgrades and roll backs, multiple variants of a package coexisting without conflict, etc.
That’s one way to look at it. The other is that these systems ignore these problems. What you’re really offering is a really good reproducible build system, at the price of rigorous specification. Reproducibility is a big deal, and there are great reasons to do this, but you can’t say the other systems are “obsolete” when they have very different goals.
What Nix/Guix are not doing is reducing the burden of specification. You write down all the details of a very specific, reproducible software stack, but you do not make that software much more composable or extensible than it already was. The user can install the version combination that you packaged, but can they easily try their own? Can they easily try to build with a different compiler/compiler version/set of compiler flags/dependency version/etc.?
npm, pip, etc. hide a lot of details from the user, and that is why people likely continue to use them (i.e., they are not “obsolete”). Constraint solving, etc., are still necessary to hide a lot of the complexity that users don’t want to deal with. Nix and Guix are great for reproducing a snapshot, but app devs want to explore the build space more than that.
Spack attempts to find a happy medium. We cryptographically hash builds, but we also let the user build things that the original package author may not have tried yet, without modifying the package. That does require all the constraint solving nastiness, but it doesn’t kill reproducibility. Spack stores the generated build provenance after doing some constraint solving [3], but the tool helps fill in the missing details of the dependency graph, not the human. Nix and Guix, AFAIK, do not do that. Spack isn’t fully reproducible because it’s not a full system package manager and it doesn’t use chroot, but you can try the same build again using the provenance that was generated when all the solving was done. The tool just helped the user along the way, and I think that’s still a very useful (not obsolete) thing.
[1] https://www.computer.org/csdl/proceedings/sc/2015/3723/00/28...
[2] Slides 9-11: https://tgamblin.github.io/files/Gamblin-Spack-SC15-Talk.pdf
[3] Slide 14: https://tgamblin.github.io/files/Gamblin-Spack-SC15-Talk.pdf