Flakes aren't real and cannot hurt you: using Nix flakes the non-flake way
jade.fyi
jade.fyi
Eventhough I've used nix for years now, I have no deep understanding of nix and I'm a very curious person who always tries to understand how stuff actually works. Nix makes me feel really dumb. It makes me feel like when I was twelve and I "programmed" by cluelessly stitching together code examples from online tutorials. I don't really understand what's going on outside of the really common use cases for nix.
I read the entire nix docs but I think it's the kind of documentation that's only useful if you already kind of get what's going on. It feels like you have to gather the knowledge from various random blogs. Last time I checked there surprisingly was not a single book which covers nix. I feel like a comprehensive book which teaches nix from the ground up and explains all the different ways of using nix would really help me and I would purchase such a book without hesitation if it existed.
Edit, I put in the request if you feel like upvoting: [0]
[0] https://www.reddit.com/r/3Blue1Brown/comments/133zoxd/commen...
Nowadays, I just use nix (the package manager) on top of PopOS.
Maybe I should give it another shot if ChatGPT can provide better guidance than reading a dozen blog posts, 11 of which are out of date.
Coming from Arch, almost every package I want is on NixOS. The couple I can remember that weren't there are Android Studio for Platform (I packaged it myself following the derivation for the regular Android Studio), Obsidian.nvim (which was packaged a week or two after I mentioned it), and there's been a couple smaller tools that I can't remember.
NixOS is great if you never tread from the happy path but it seems most of my work requires tools way off the happy path.
Typically these are troubleshooting a flake for a project. I think it works because flakes are nice and self contained: you can paste the whole flake. I find it does significantly worse when trying to help with contributions to nixpkgs, presumably because the context is not bite sized.
Even so. Removing the packages you talk about Nix is still largest. So again not misleading.
Most Haskell packages are not included in the AUR and haskellPackages alone makes up over 17,000 packages on nixpkgs compared to ~1,100 on AUR - subtract just that and Arch pulls ahead. Nixpkgs also has almost twice as many Python packages because python310Packages is separate from python311Packages.
I bet there are lots of other non-language examples too: a quick search found emacsPackages which contains six thousand packages.
You have to prove that it has less of the software available on other distros. Even if you could find a billion small packages that represents only a single package in every distro this is not evidence or a proof that support your claim.
Simply put, you need to show where nix does not have a larger number of packages that are available on other distros.
Given that nix is mostly automated and pulls from the same root sources as every other distro you are going to be hard pressed to find a significant number of cases where nix lacks a package.
Not misleading.
I think nixpkgs is probably so much larger for two reasons:
- running unpackaged software on nixos is painful. - it’s very easy to contribute to nixpkgs (standard github pr to a monorepo).
It makes little sense if you compare it like this. It would be stupid for any other package repo to include those package since they already exist. The only reason nixpkgs needs to do it is for purity reasons.
The metric that matters is "how many packages can I easily install on this distro". And nixpkgs most certainly is not in the lead here.
If you want a non-misleading comparison you have to remove all these packages, which are actually managed by a different package manager every. from nixpkgs and put that into repology.
https://discourse.nixos.org/t/github-numtide-system-manager-...
The idea is to provide some of the NixOS whole-system management niceties (especially controlling systemd units) but on non-NixOS systems like Ubuntu, in a way that behaves more like puppet/chef/ansible rather than competing with the native package manager.
Nix is fundamentally just a build system. It's based around `.drv` files ("derivations", which are like a Makefile rule). These are text files containing a system type (e.g. x86_64-linux), the path to an executable, a list of string arguments to give it and a key/value map of strings to use as environment variables. Those basically form a declarative version of an `exec` syscall.
Each .drv file also specifies some paths as its "outputs", and may optionally specify some "inputs": an input is either a raw file path, or a pair of [path to another .drv file, name of output].
The basic operation in Nix is a "build", like `nix-build /nix/store/...foo.drv`, which tells Nix to produce that derivation's output paths. Nix does this as follows:
- Look in that .drv file to find its output paths. If they already exist on the filesystem, print the paths and halt.
- If some of the output paths don't exist, query any configured binary caches to see if they have those paths. If so: download them, print the paths and halt.
- If some outputs don't exist, and aren't in any configured cache, then Nix will need to execute the derivation's command...
- First Nix checks that all the inputs referenced as raw files exist; if not, it exits with an error.
- Next Nix checks whether all of the inputs which reference other .drv files exist. Any which don't exist will first be built by following these steps (and so on, recursively).
- Once all of the "inputs" exist, Nix will run the derivation's executable (maybe in a sandbox, but that's not important)
- When the executable has finished, Nix will check whether all of its output paths now exist. If not, it prints an error message and exits. If they do exist, it prints the paths and halts.
That's the core idea of what Nix is doing: it's a way to specify individual command invocations, create dependencies between commands using the paths of their outputs, and allowing parts of such dependency trees to be skipped by downloading the desired outputs directly from a cache instead. It's like Make, but spread across multiple files.
The "magic trick" that makes Nix special is that the path of every .drv file must contain a hash of its contents. Since .drv files can reference each other by their paths, these hashes form a Merkle tree which completely specifies the entire dependency graph of each derivation (similar to how a git commit includes the commit ID of its parents, and hence specifies its entire history).
This is a remarkably powerful and general approach to specifying the contents of files/folders. The downside is that it's tedious to write all of these .drv files by hand (especially calculating the hashes), so Nix comes with a simple scripting language to generate them. We can use the -E argument of nix-build to specify a "Nix expression", which will generate a .drv file to build (the `derivation` function will default to specifying a single output called "out"):
$ nix-build -E 'derivation { name = "myName"; system = "aarch64-linux"; builder = "/bin/sh"; args = [ "-c" "echo foo 1>&2 && echo bar > $out" ]; }'
this derivation will be built:
/nix/store/irxpvx4cs8jmyysrd5hhqjx66blg4b8k-myName.drv
building '/nix/store/irxpvx4cs8jmyysrd5hhqjx66blg4b8k-myName.drv'...
foo
/nix/store/mdaw1ajdd6zp8dnbcbxvay4lw9dssaqp-myName
$ cat /nix/store/mdaw1ajdd6zp8dnbcbxvay4lw9dssaqp-myName
bar
$ cat /nix/store/irxpvx4cs8jmyysrd5hhqjx66blg4b8k-myName.drv
Derive([("out","/nix/store/mdaw1ajdd6zp8dnbcbxvay4lw9dssaqp-myName","","")],[],[],"aarch64-linux","/bin/sh",["-c","echo foo 1>&2 && echo bar > $out"],[("builder","/bin/sh"),("name","myName"),("out","/nix/store/mdaw1ajdd6zp8dnbcbxvay4lw9dssaqp-myName"),("system","aarch64-linux")])
I love Nix. I use it every day to manage multiple local dev environments. And I use devenv instead of docker for sharing project-specific environments with others.
Who wants to use docker on a Mac anyway?
"Super Fast Emacs: Bleeding edge Emacs that fixes itself, thanks to a community overlay"
Could you possibly tell me (or link to the explanation) what's special about that Emacs instance?
https://github.com/dustinlyons/nixos-config/blob/main/module... -- This file appears to be part of it, but I'm not very experienced with Nix so I am unsure of how to find the file that says: "install Emacs by doing this, that, and the other".
I use this homebrew cask and have been very happy with it thus far, but I'm always up for some new exploration. https://github.com/d12frosted/homebrew-emacs-plus
EDIT2: oh lordy, this snippet:
(setq org-todo-keyword-faces
`(("TODO" . ,dl/green-color)
("STARTED" . ,dl/yellow-color)
("WAITING" . ,dl/light-grey-color)
("DELEGATED" . ,dl/teal-color)
("DEFERRED" . ,dl/dark-grey-color)
("SOMEDAY" . ,dl/purple-color)
("DONE" . ,dl/dark-grey-color)
("CANCELED" . ,dl/dark-grey-color)
("PROJECT" . ,dl/blue-color)
("APPT" . ,dl/green-color)))
oh and the links at the top of that file are 404s.
What that means is if something is broken in Emacs, the community will fix it, and all I need to do is run `nix flake update` to grab the latest commit and then `nix run .#build-switch` to alter my system. Easy.
Thanks for the heads-up on the 404s! I've fixed those links.
In re: to org-agenda, I don't use that as much anymore. But I heavily, heavily using org-roam w/ org-roam-dailies everyday to build my own networked graph of notes. For tasks, nowadays I just use simple docs for projects and Asana to keep a catalog of everything.
For a long time, before really doubling down on Nix, Docker Desktop was my go-to for getting a Linux runtime whenever MacOS wasn’t sufficient for what I needed. It was slow, but way more portable than having a Linux VM running all the time.
Fast forward to now (since I’ve really invested in Nix) and Nix+Nix-Darwin makes my Mac feel just as configurable and extensible as NixOS/any Linux distribution.
Nix makes me an extremely content MacOS user. :)
Keeping my NixOS machine on my desk and Macbook in sync seamlessly is also super cool.
I have a similar albiet differently-structured guide at https://github.com/drakerossman/nixos-musings, but nowhere near the amount of stars you have.
2. Use your README to grab the attention of the user. I have a list of features, videos, etc. at the top.
3. I regularly post on various subreddits, Hacker News, etc. trying to support new Nix users. Sometimes I link to my project as an example of code that may help them. I've done this over a couple of years, so there's now a collection of posts people are finding and driving traffic.
4. My experience was that in the beginning, I had to grind out each star one by one. The project never went "viral". But after a certain growth point, Github will pick it up and start recommending it to others.
5. Reddit and Github are now my biggest sources of traffic.
Hope this helps!
Last time I tried Nix on MacOS it did not end well. Maybe I’ll try this in a VM and see how it goes…
Flake-less Nix handles this just fine.
Flakes are why I stopped and likely won't go back. I tried to add a flake that contained PragmataPro (a paid for font). Having a git repo was easy I just wouldn't push it anywhere to avoid copyright issues.
Figuring out the syntax to actually get the font files installed was maddening. It probably didn't help that I followed best practices and built some crazy cross compilation mechanism into my flake but it was almost impossible to figure out how to extract a zip file and then copy to the final destination. Well unless I downloaded the zip file then it was trivial...
Avoiding flakes except as an isolation point likely would have allowed me to stay engaged with the project. Might have to check this out if I ever feel the need again.
Yeah looks like callPackage is what I needed.
I think you could get a lot out of that google course teaching technical writing. There were some other HN posts on technical writing recently too. You’re a technical writer now, if you didn’t know already, so it would be worth learning how to apply that skill well. You’ll get a lot more leverage for every input.
I unfortunately have very little time to edit the posts, and for the stuff not yet posted, I do not like to post the drafts. And I have tons (tons!!!) of such drafts. A promise to myself I made is that I am going to have a good cadence of posting for this year.
There are definitely more articles to come, till end of January for sure, and if you've found the stuff useful/intersting, you may would want to subscribe on some medium I use (say Twitter, or Mastodon, links on my website).
I would say that if you slow down, learn more about how to communicate technical content, and consciously practice those skills, then you'll achieve both your immediate goal and gain a really useful life skill. Just my take! Best wishes on the journey. Oh and GPT4 could massively help with this stuff now too, so you may be able to just keep cranking it out like you already do, but get the bot to rewrite and properly structure it all. Many options.
Most package managers exist independently from the packages they manage. Nix integrates them directly into its monolithic source tree.
So when you ask nix to install a package, what version does it give you?
The original method is channels: each channel follows a branch of the NixOS/nixpkgs GitHub repo. Every package is given a global name in nixpkgs/pkgs/top-level/all-packages.nix
This is...a mess. Versions? They go in the name sometimes, like python2. Names aren't consistent everywhere, either. The name of a package in all-packages.nix might need to be different from the name in its own derivation source. Good luck finding it.
Not every package gets a global name, either. Some are moved into separate attributes sets. Some get both.
Figuring out what magic words need to go into your configuration.nix (or at the end of a command) is a lot more difficult than it should be. Even more difficult is figuring out what needs to go in a new package.
Flakes try to dodge this mess by moving it out of the global namespace, out of the nixpkgs source tree, and into the source of the thing you want packaged.
I'm not a huge fan.
A couple of minor points: the Nix package collection lives in the Nixpkgs repo. The package manager's source has its own repo.
> So when you ask nix to install a package, what version does it give you?
> The original method is channels: each channel follows a branch of the NixOS/nixpkgs GitHub repo. Every package is given a global name in nixpkgs/pkgs/top-level/all-packages.nix
> This is...a mess. Versions?
The monorepo of package declarations (build recipes) is traditional for source-based package managers like Nix. Anyway, it doesn't preclude versioning in a more traditional way. Gentoo's Portage, for instance, has a tree of build recipes but also supports multiple versions, in-tree, in a first-class way. Maybe Nixpkgs or a similar package set could have something like that some day.
Overall, I agree that the namespacing in Nixpkgs is messy and also that replacing Nixpkgs with assorted flakes that refer to one another as inputs doesn't seem great.
There are two reasons I'm really frustrated with the namespace problem:
1. Discoverability
I have spent a very significant amount of time just trying to answer the simple question, "what is the package named?" The worst part is when the package does have an obvious top-level/all-packages.nix name, but that isn't the one you are supposed to use! Sometimes, a package will need to change more of your environment than PATH, like udev rules or systemd units. In that case, it will have a totally different attribute like services.openssh.enable; but there is no clear way to learn this is the case, or what the attribute is! I still don't have a good sense of how nixpkgs is organized, and I've been using NixOS as a daily driver for 5 years.
2. Flexibility
Writing a new derivation (package) is way too difficult. Documentation is improving, but it's still not nearly good enough. There are just too many layers, and too much magic. God forbid you need a different version of a library. Of course, that's the problem that flakes solve, but it's still hard to find what you need, even when it's there.
Like update-alternatives. Like simple ways to reinstall the system while keeping /home. Like a file with all installed packages that you can edit to install/uninstall. Like nicer merging of overriden config when a new default file is available from upstream.
And I don't mind reading 'code as docs' in general but it's pretty much required here. Mostly the option names are pretty understandable, and it's helpful to see in the code exactly what it does, not just a human version.
But truly invaluable is sourcegraph (I guess github search is better but I still don't like it) so you can search through other peoples config https://sourcegraph.com/search?q=context:global+file:.*nix+B...
On the other hand, I wanted to have emacs compiled with sqlite (needed for magit) and it took me hours to get working, then had to keep modifying to pin the version so it didn't recompile constantly
We've gone pretty all in on Flakes with our Devbox product[1] -- We think they simplify a lot of confusion on how to package and use Nix once you embrace them. I hope they become a more accepted standard in the future.
[0]: https://youtu.be/v7lIGYU0onA?si=3Wpbx82GcZVxwKY6&t=5256 [1]: https://www.jetpack.io/blog/why-did-nix-adopt-flakes/
On Debian pinning is a little strange. You can try `apt install foo=1.2.3` and it will work until the next versions is released. Debian removes old versions from the mirrors. However they operate a snapshot archive with all the old package versions, so reproducible builds can use that and should continue to work over many years.
Does Nix have a way to ensure pinning continues to work over many years?
It doesn't, actually - all the fetchers are [fixed-output derivations](https://nixos.org/manual/nix/stable/language/advanced-attrib...) where you specify not only the source, but the hash it is expected to produce. And so the fetched files will be hashed and placed in the nix store under that hash. And therefore subsequent builds using that artifact will use the already-stored artifact and not care if the source is reachable or not. You know that all "external" inputs will be captured by fixed-output derivations, because any derivation that isn't fixed-output is blocked from network access.
For anything built by the hydra CI, this will be retained on central cache.nixos.org. Or you can copy from a store of some machine that has it.
Now, if it's something not on the official cache.nixos.org, and the source is gone, and you've lost (or pruned) your store, you will need to get the file from somewhere and put it back in your store: https://nixos.org/manual/nix/stable/command-ref/nix-store/ad.... Or you just change the source URL (this won't change the derivations depending on it, because the input being transferred is identified by the hash that is to be delivered, not by the URL it comes from).
If I'm following correctly, this would prevent issues like builds failing due to the leftpad package getting unpublished, since you'd use the cache/archive instead of failing to fetch the missing package.
> and you've lost (or pruned) your store
It's common for academic code, for example, to be built once, published and then shelved. The "you" in "your store" ends up being different people over time.
Apt generally has this issue too, while Debian keeps old packages around (snapshot archive), third parties origins and other apt based systems may not.
I'm going to explore nix more, thank you!
Also discussing flakes in some other articles of mine.
Plus, you have to jump through hoops to install non-free software, which goes against the ethos of Guix anyway.
IMHO, Nix is clearly "the winner" here and we'll see more and more adoption as it improves. Lots of folks are doing exciting work (see https://determinate.systems/, https://devenv.sh/, https://flakehub.com/). And the scale and organization around nixpkgs is damn impressive.
Guix (deliberately) doesn't support anything like the NixOS module semantics. 'Services' are its alternative, of course. I don't know if there is a capability for per-project service declaration in Guix atm, but it wouldn't surprise me!
Anyone can create their packages channel or import existing
Also Interesting that you can now install Guix in Nix, or Nix in Guix, or both in any other distro
So you can easily install any package from Nix
There is a lot of work but flakes are still too confusing
So winning is relative
That it's not indicates that there is a reason why it's not, and I don't think it's because of a lack of demand. Rather, I think there's something about Nix that causes people to bounce off it hard, and if people want Nix to be more widely used, then it absolutely is a problem that aficionados of Nix need to look at and address.
But I'd say the biggest issue with Nix in general is it's lack of discoverability. This is evidenced in the number of third-party tools built around helping figure out what even exists (noogle.dev and search.nixos.org for example). The language is so dynamic, trying to figure out where things are coming from becomes difficult. Simply put, there's too much magic.
Nix also has a tendency to drop down into massive bash scripts to actually do stuff. I really like guix as an alternative in theory, but I haven't had much luck using it yet.
And yet... there's so much I just don't understand. Doing simple overlays is relatively easy, but anything of moderate complexity really becomes frustrating.
It makes it quite difficult to recommend NixOS when the chasm between being able to use it vs being able to understand it is so wide.
There is no turnkey solution for this, but there is a way out and a way to (eventually) tame the complexity.
Nix is fantastic. I'm writing this comment on a laptop that has only ever run NixOS.
Nix is also painful to learn. It's painful to use.
That doesn't mean it's impossible, or even that it isn't worth it.
It means that Nix has room to improve.
As I've gotten deeper into using Terraform at work, I've found it much, much more painful than Nix— so much so that it has me questioning whether I want to continue to work with cloud infrastructure at all. Yet Terraform is 'industry standard' and widely praised, where Nix is marginal and maligned for its difficulty.
Maybe it's because people think of NixOS more than Nix in all its broad uses, and when they do so they compare NixOS to desktop operating systems where those people don't see themselves as very interested in configuration management.
But I wish that many industry standard tools had comparable iteration times to Nix, or concise languages like Nix, reliable transitions between configurations like Nix, efficient caching like Nix, minimally destructive replacement of old resources like Nix, etc.
Most of the things people call 'immutable infrastructure' today are painfully slow and transparently mutable. I'm much happier when I'm reliably iterating on local Nix stuff than when I'm praying that `terraform plan`'s success isn't a fluke this time, and that the corresponding `terraform apply` will actually succeed. I'd rather spend an hour on the the thorniest Nix issues than spend 30 minutes fucking around trying to 'help' Terraform delete a resource that requires manual intervention to actually delete, for some godforsaken reason.
Everything I did to alter and configure my OS is self-contained and self-documented in one file.
I can try anything without ever risking the stability of my system. If it breaks, I just reboot, and pick the second most recent GRUB entry.
If I ever want to change my root filesystem or move to a new disk, I can do the whole reinstall in a few short obvious steps.
I can keep my laptop and my desktop configurations identical. If I want a few things to be different, they can go in a separate config file.
The trickiest part is running software that isn't already packaged. It's usually not too hard, though: there are a few nix features that construct a more traditional/compatible environment, like steam-run and nix-ld. The biggest problem is that their existence isn't obvious to new users. They should really go near the top of the manual.