Nix journey part 0: Learning and reference materials | Hacker News Reader

Nix journey part 0: Learning and reference materials

tinkering.xyz

140 pointsrrampage76 comments

The nix package search website is OK, but it doesn't let you filter by the names of installed binaries. A lot of the time, you have a question like "what nixpkgs attribute do i install in order to get the `python3` command". I recently wrote a command line tool that allows you to do this. It uses the same elasticsearch index as the search website, but allows more powerful filtering. If anyone is thinking of getting into nix, please consider trying it out!

https://github.com/peterldowns/nix-search-cli

I've also noticed this. If you install `rustc` it says that you get `rust-gdb`, but that's really just a wrapper script around `gdb`, which it doesn't install.

And vice versa. It's tricky to know what the binary will actually be named.

I have had to look directly in the /nix/store enough times that I lost count.

I will say this is not a nix specific problem and also not trivially solvable by policy because of packages that contain multiple binaries as produced by the upstream developer (e.g. grep/fgrep/egrep, pacman/makepkg/pacman-key, , go/gofmt, ffmpeg/ffprobe, qemu has about a billion, etc.).

It would be trivial to print the contents of the nix store that package uses.

It would be pretty straightforward to show only what ends up in the path.

Cool!

I love Nix, but really am not a fan of the Nix package listing website. So, it is nice to see.

And I guess worth mentioning, but if you or anyone else is ever feeling a little less lazy than me and want to add more available metadata, some of my main gripes with the official site:

- No easy way to see dependencies without viewing the package derivation source code.

- No easy way to see "reverse" dependencies, period, as far as I am aware, short of basically grepping the entire nixpkgs repo, and even then you won't get a totally accurate answer.

This seems like it'd be awful to implement given I don't think you can realistically know this information without evaluating the entirety of `all-packages.nix`. But it is something I miss about stuff like Arch Linux's package website. And I think in some ways, it is even more important with stuff like Nix if you intend to override a particular package globally, since you may end up needing to rebuild god knows how many things as a result.

- No easy way to filter by package types, outside of the very small set of existing package sets (e.g., `vimPackages`, `mpvScripts, etc).

As an example, I'd really like to be able to see all font packages in the nixpkgs repo, but there's just no nice way to do this easily, unlike some other package managers. It would be another kind of awful feature to implement, since there is not currently any grouping that I am aware of.

You /can/ kind of grep the `all-packages.nix` file and get a rough idea by searching for what packages call `./data/fonts`, but actually parsing that is pretty rough, since not everything is as simple as `font-name = callPackage ./data/fonts/font-name {};`

A good example of that parsing annoyance would be the noto-fonts [1].

- No easy way to see build options. For example, packages like `mpv` or `ffmpeg_5` have loads of different feature flags, some of which are unexpectedly disabled or enabled.

Again, I think this would be not fun to implement. But it is something that I think Gentoo gets really right with the listing of 'use flags' on their package listings [2] (in fact, Gentoo's site kind of gets everything right; it feels like the gold standard to me). NetBSD/pkgsrc is another one of the few that does well, albeit with a less-fancy design [3].

[1]: <https://github.com/NixOS/nixpkgs/blob/96c0fdeb6accc9f76f1887...>

[2]: <https://packages.gentoo.org/packages/media-video/ffmpeg>

[3]: <https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/multimedia/...>

You may want to submit this feedback as an issue to the nixos-search repository, where they have the indexing code and website interface. I have no idea who is in charge of Nix stuff and I find it all horrible to use, totally agreed with you that all basic package manager things you'd want to be able to do are impossible today with nix.

https://github.com/NixOS/nixos-search

> No easy way to see dependencies without viewing the package derivation source code.

nix-store -qR `which python`

> No easy way to see "reverse" dependencies, period, as far as I am aware, short of basically grepping the entire nixpkgs repo, and even then you won't get a totally accurate answer.

nix-store -q --referrers `which python`

This and many other querying commands are explained in the Command Reference of Nix manual. Have you read it? [1]

[1] https://nixos.org/manual/nix/stable/command-ref/nix-store.ht...

I've used nix-index to answer this question:

         $ nix-locate --top-level /bin/python3
    remarkable2-toolchain.out                             0 s /nix/store/lygxp772hm743v33wbri22f2k78k1vsb-remarkable2-toolchain-3.1.2/sysroots/cortexa7hf-neon-remarkable-linux-gnueabi/usr/bin/python3
    remarkable2-toolchain.out                             0 s /nix/store/lygxp772hm743v33wbri22f2k78k1vsb-remarkable2-toolchain-3.1.2/sysroots/cortexa7hf-neon-remarkable-linux-gnueabi/usr/bin/python3-config
    remarkable2-toolchain.out                         5,544 x /nix/store/lygxp772hm743v33wbri22f2k78k1vsb-remarkable2-toolchain-3.1.2/sysroots/cortexa7hf-neon-remarkable-linux-gnueabi/usr/bin/python3.8
    remarkable2-toolchain.out                             0 s /nix/store/lygxp772hm743v33wbri22f2k78k1vsb-remarkable2-toolchain-3.1.2/sysroots/cortexa7hf-neon-remarkable-linux-gnueabi/usr/bin/python3.8-config
    remarkable2-toolchain.out                         2,114 x /nix/store/lygxp772hm743v33wbri22f2k78k1vsb-remarkable2-toolchain-3.1.2/sysroots/cortexa7hf-neon-remarkable-linux-gnueabi/usr/bin/python3.8-config-lib
    remarkable2-toolchain.out                             0 s /nix/store/lygxp772hm743v33wbri22f2k78k1vsb-remarkable2-toolchain-3.1.2/sysroots/x86_64-codexsdk-linux/usr/bin/python3
    remarkable2-toolchain.out                           306 x /nix/store/lygxp772hm743v33wbri22f2k78k1vsb-remarkable2-toolchain-3.1.2/sysroots/x86_64-codexsdk-linux/usr/bin/python3.8
    remarkable2-toolchain.out                        18,432 x /nix/store/lygxp772hm743v33wbri22f2k78k1vsb-remarkable2-toolchain-3.1.2/sysroots/x86_64-codexsdk-linux/usr/bin/python3.8.real
    remarkable-toolchain.out                              0 s /nix/store/qk2p0ahd3nndi2zlwm9d0psahlrpikm0-remarkable-toolchain-3.1.2/sysroots/cortexa9hf-neon-remarkable-linux-gnueabi/usr/bin/python3
    remarkable-toolchain.out                              0 s /nix/store/qk2p0ahd3nndi2zlwm9d0psahlrpikm0-remarkable-toolchain-3.1.2/sysroots/cortexa9hf-neon-remarkable-linux-gnueabi/usr/bin/python3-config
    remarkable-toolchain.out                          5,544 x /nix/store/qk2p0ahd3nndi2zlwm9d0psahlrpikm0-remarkable-toolchain-3.1.2/sysroots/cortexa9hf-neon-remarkable-linux-gnueabi/usr/bin/python3.8
    remarkable-toolchain.out                              0 s /nix/store/qk2p0ahd3nndi2zlwm9d0psahlrpikm0-remarkable-toolchain-3.1.2/sysroots/cortexa9hf-neon-remarkable-linux-gnueabi/usr/bin/python3.8-config
    remarkable-toolchain.out                          2,114 x /nix/store/qk2p0ahd3nndi2zlwm9d0psahlrpikm0-remarkable-toolchain-3.1.2/sysroots/cortexa9hf-neon-remarkable-linux-gnueabi/usr/bin/python3.8-config-lib
    remarkable-toolchain.out                              0 s /nix/store/qk2p0ahd3nndi2zlwm9d0psahlrpikm0-remarkable-toolchain-3.1.2/sysroots/x86_64-codexsdk-linux/usr/bin/python3
    remarkable-toolchain.out                            306 x /nix/store/qk2p0ahd3nndi2zlwm9d0psahlrpikm0-remarkable-toolchain-3.1.2/sysroots/x86_64-codexsdk-linux/usr/bin/python3.8
    remarkable-toolchain.out                         18,432 x /nix/store/qk2p0ahd3nndi2zlwm9d0psahlrpikm0-remarkable-toolchain-3.1.2/sysroots/x86_64-codexsdk-linux/usr/bin/python3.8.real
    qtile.out                                        15,872 x /nix/store/mrpll8c98mvnnzab5znqawz56dvq841j-qtile-0.22.1/bin/python3
    qtile.out                                        15,872 x /nix/store/mrpll8c98mvnnzab5znqawz56dvq841j-qtile-0.22.1/bin/python3-config
    qtile.out                                        15,872 x /nix/store/mrpll8c98mvnnzab5znqawz56dvq841j-qtile-0.22.1/bin/python3.10
    qtile.out                                        15,872 x /nix/store/mrpll8c98mvnnzab5znqawz56dvq841j-qtile-0.22.1/bin/python3.10-config
    python3Minimal.out                                    0 s /nix/store/igdcckxrd115041zvvcqmw9qxwf6pfxv-python3-minimal-3.10.8/bin/python3
    python3Minimal.out                               15,616 x /nix/store/igdcckxrd115041zvvcqmw9qxwf6pfxv-python3-minimal-3.10.8/bin/python3.10
    python311.out                                         0 s /nix/store/ipaj21xdmnw2f2bs9mcd7czkzhdhbcfv-python3-3.11.1/bin/python3
    python311.out                                         0 s /nix/store/ipaj21xdmnw2f2bs9mcd7czkzhdhbcfv-python3-3.11.1/bin/python3-config
    python311.out                                    15,616 x /nix/store/ipaj21xdmnw2f2bs9mcd7czkzhdhbcfv-python3-3.11.1/bin/python3.11
    python311.out                                     3,404 x /nix/store/ipaj21xdmnw2f2bs9mcd7czkzhdhbcfv-python3-3.11.1/bin/python3.11-config
    python38.out                                          0 s /nix/store/n3ma7kwpssmwlgv118pc6k5ia0splp2m-python3-3.8.16/bin/python3
    python38.out                                          0 s /nix/store/n3ma7kwpssmwlgv118pc6k5ia0splp2m-python3-3.8.16/bin/python3-config
    python38.out                                     15,616 x /nix/store/n3ma7kwpssmwlgv118pc6k5ia0splp2m-python3-3.8.16/bin/python3.8
    python38.out                                      3,444 x /nix/store/n3ma7kwpssmwlgv118pc6k5ia0splp2m-python3-3.8.16/bin/python3.8-config
    python37.out                                          0 s /nix/store/a1gzdpyn8bbp192cavnxl6sa3snjx563-python3-3.7.16/bin/python3
    python37.out                                          0 s /nix/store/a1gzdpyn8bbp192cavnxl6sa3snjx563-python3-3.7.16/bin/python3-config
    python37.out                                     15,616 x /nix/store/a1gzdpyn8bbp192cavnxl6sa3snjx563-python3-3.7.16/bin/python3.7
    python37.out                                          0 s /nix/store/a1gzdpyn8bbp192cavnxl6sa3snjx563-python3-3.7.16/bin/python3.7-config
    python37.out                                     15,616 x /nix/store/a1gzdpyn8bbp192cavnxl6sa3snjx563-python3-3.7.16/bin/python3.7m
    python37.out                                      3,210 x /nix/store/a1gzdpyn8bbp192cavnxl6sa3snjx563-python3-3.7.16/bin/python3.7m-config
    python3Full.out                                       0 s /nix/store/rf7mdhqwpfzj239ylwim4rfqhw4fz52h-python3-3.10.8/bin/python3
    python3Full.out                                       0 s /nix/store/rf7mdhqwpfzj239ylwim4rfqhw4fz52h-python3-3.10.8/bin/python3-config
    python3Full.out                                  15,616 x /nix/store/rf7mdhqwpfzj239ylwim4rfqhw4fz52h-python3-3.10.8/bin/python3.10
    python3Full.out                                   3,447 x /nix/store/rf7mdhqwpfzj239ylwim4rfqhw4fz52h-python3-3.10.8/bin/python3.10-config
    python38Full.out                                      0 s /nix/store/b5yxnf8vq4rdkbz1z8wydhcxz3r1zsg7-python3-3.8.16/bin/python3
    python38Full.out                                      0 s /nix/store/b5yxnf8vq4rdkbz1z8wydhcxz3r1zsg7-python3-3.8.16/bin/python3-config
    python38Full.out                                 15,616 x /nix/store/b5yxnf8vq4rdkbz1z8wydhcxz3r1zsg7-python3-3.8.16/bin/python3.8
    python38Full.out                                  3,444 x /nix/store/b5yxnf8vq4rdkbz1z8wydhcxz3r1zsg7-python3-3.8.16/bin/python3.8-config
    sourcehut.python.out                                  0 s /nix/store/lbn7f0d2k36i4bgfdrjdwj7npy3r3h5d-python3-3.10.8/bin/python3
    sourcehut.python.out                                  0 s /nix/store/lbn7f0d2k36i4bgfdrjdwj7npy3r3h5d-python3-3.10.8/bin/python3-config
    sourcehut.python.out                             15,616 x /nix/store/lbn7f0d2k36i4bgfdrjdwj7npy3r3h5d-python3-3.10.8/bin/python3.10
    sourcehut.python.out                              3,447 x /nix/store/lbn7f0d2k36i4bgfdrjdwj7npy3r3h5d-python3-3.10.8/bin/python3.10-config
    python39Full.out                                      0 s /nix/store/ckxlmknrfn6c55jwnw86ha9440w4a9ff-python3-3.9.15/bin/python3
    python39Full.out                                      0 s /nix/store/ckxlmknrfn6c55jwnw86ha9440w4a9ff-python3-3.9.15/bin/python3-config
    python39Full.out                                 15,616 x /nix/store/ckxlmknrfn6c55jwnw86ha9440w4a9ff-python3-3.9.15/bin/python3.9
    python39Full.out                                  3,444 x /nix/store/ckxlmknrfn6c55jwnw86ha9440w4a9ff-python3-3.9.15/bin/python3.9-config
    python39.out                                          0 s /nix/store/hm6bqikghylrwgvqd42lgghjdq9p3hac-python3-3.9.15/bin/python3
    python39.out                                          0 s /nix/store/hm6bqikghylrwgvqd42lgghjdq9p3hac-python3-3.9.15/bin/python3-config
    python39.out                                     15,616 x /nix/store/hm6bqikghylrwgvqd42lgghjdq9p3hac-python3-3.9.15/bin/python3.9
    python39.out                                      3,444 x /nix/store/hm6bqikghylrwgvqd42lgghjdq9p3hac-python3-3.9.15/bin/python3.9-config
    python37Full.out                                      0 s /nix/store/ys4my3x7wwb6r6mwfkyk6q9pay2jm3dz-python3-3.7.16/bin/python3
    python37Full.out                                      0 s /nix/store/ys4my3x7wwb6r6mwfkyk6q9pay2jm3dz-python3-3.7.16/bin/python3-config
    python37Full.out                                 15,616 x /nix/store/ys4my3x7wwb6r6mwfkyk6q9pay2jm3dz-python3-3.7.16/bin/python3.7
    python37Full.out                                      0 s /nix/store/ys4my3x7wwb6r6mwfkyk6q9pay2jm3dz-python3-3.7.16/bin/python3.7-config
    python37Full.out                                 15,616 x /nix/store/ys4my3x7wwb6r6mwfkyk6q9pay2jm3dz-python3-3.7.16/bin/python3.7m
    python37Full.out                                  3,210 x /nix/store/ys4my3x7wwb6r6mwfkyk6q9pay2jm3dz-python3-3.7.16/bin/python3.7m-config
    ihaskell.out                                        294 x /nix/store/yj8p9gd2irrh0z16d3b0ljv4vps920h7-ihaskell-with-packages/bin/python3
    ihaskell.out                                        301 x /nix/store/yj8p9gd2irrh0z16d3b0ljv4vps920h7-ihaskell-with-packages/bin/python3-config
    ihaskell.out                                        297 x /nix/store/yj8p9gd2irrh0z16d3b0ljv4vps920h7-ihaskell-with-packages/bin/python3.10
    ihaskell.out                                        304 x /nix/store/yj8p9gd2irrh0z16d3b0ljv4vps920h7-ihaskell-with-packages/bin/python3.10-config
    google-cloud-sdk.out                             20,744 x /nix/store/7dw9fdkqs25fiwzq3r2z1y21gc97c8wd-google-cloud-sdk-408.0.1/google-cloud-sdk/platform/bundledpythonunix/bin/python3
    google-cloud-sdk.out                              3,254 x /nix/store/7dw9fdkqs25fiwzq3r2z1y21gc97c8wd-google-cloud-sdk-408.0.1/google-cloud-sdk/platform/bundledpythonunix/bin/python3-config
    google-cloud-sdk.out                             20,744 x /nix/store/7dw9fdkqs25fiwzq3r2z1y21gc97c8wd-google-cloud-sdk-408.0.1/google-cloud-sdk/platform/bundledpythonunix/bin/python3.9
    google-cloud-sdk.out                              3,254 x /nix/store/7dw9fdkqs25fiwzq3r2z1y21gc97c8wd-google-cloud-sdk-408.0.1/google-cloud-sdk/platform/bundledpythonunix/bin/python3.9-config
    google-cloud-sdk-gce.out                         20,744 x /nix/store/b96rpkxpw920sbbswz1xrrrpjdb1f30c-google-cloud-sdk-408.0.1/google-cloud-sdk/platform/bundledpythonunix/bin/python3
    google-cloud-sdk-gce.out                          3,254 x /nix/store/b96rpkxpw920sbbswz1xrrrpjdb1f30c-google-cloud-sdk-408.0.1/google-cloud-sdk/platform/bundledpythonunix/bin/python3-config
    google-cloud-sdk-gce.out                         20,744 x /nix/store/b96rpkxpw920sbbswz1xrrrpjdb1f30c-google-cloud-sdk-408.0.1/google-cloud-sdk/platform/bundledpythonunix/bin/python3.9
    google-cloud-sdk-gce.out                          3,254 x

Previous comment is getting long, but if you want to get rid of things like "googlecloud-sdk/platform/bundledpythonunix/bin" you can use:

    nix-locate --top-level --regex '^/bin/python3'

Which will only show packages with "/bin/python3" in the root.

Most of the time this is unnecessary, but since python gets vendored in a lot of packages, it comes up with this example.

I can't tell if you're joking, but I'm not sure how these results told you anything. I find nix-index and nix-locate entirely unusable.

Not joking, I just scan through the first column. If you want to not have to do so, adding -1 cleans it up:

    $ nix-locate --top-level -1 --regex '^/bin/python3'
    qtile.out
    python3Minimal.out
    python311.out
    python38.out
    python37.out
    python3Full.out
    python38Full.out
    sourcehut.python.out
    python39Full.out
    python39.out
    python37Full.out
    ihaskell.out

Also things like "python3" and "gcc" and such are going to show way more hits than, for example, "firefox":

    $ nix-locate --top-level -1 --regex '^/bin/firefox'
    firefox-esr-102-unwrapped.out
    firefox_decrypt.out
    firefox-unwrapped.out
    firefox.out

Here, I made a short shell script to show all the results with package descriptions (requires enabling the "nix" command):

    #!/bin/sh
    results="$(nix-locate -1 --regex --top-level "^/bin/${1}\$")"
    for item in $results; do
        nix search "nixpkgs#${item%.*}"
    done

Example:

    $ ./nix-find-executable.sh python3
    * legacyPackages.x86_64-linux.qtile (0.22.1)
      A small, flexible, scriptable tiling window manager written in Python
    * legacyPackages.x86_64-linux.python3Minimal (3.10.8)
      A high-level dynamically-typed programming language
    * legacyPackages.x86_64-linux.python311 (3.11.1)
      A high-level dynamically-typed programming language
    * legacyPackages.x86_64-linux.python38 (3.8.16)
      A high-level dynamically-typed programming language
    * legacyPackages.x86_64-linux.python37 (3.7.16)
      A high-level dynamically-typed programming language
    * legacyPackages.x86_64-linux.python3Full (3.10.8)
      A high-level dynamically-typed programming language
    * legacyPackages.x86_64-linux.python38Full (3.8.16)
      A high-level dynamically-typed programming language
    * legacyPackages.x86_64-linux.sourcehut.python (3.10.8)
      A high-level dynamically-typed programming language
    * legacyPackages.x86_64-linux.python39Full (3.9.15)
      A high-level dynamically-typed programming language
    * legacyPackages.x86_64-linux.python39 (3.9.15)
      A high-level dynamically-typed programming language
    * legacyPackages.x86_64-linux.python37Full (3.7.16)
      A high-level dynamically-typed programming language
    * legacyPackages.x86_64-linux.ihaskell

Relatedly, check out comma. It's basically a shortcut prefix command that will search packages for the binary you want to run (via nix-index), and gives you an interactive choice if there are multiple. Which package was drill in again? No matter, I'll just prefix a comma :)

https://github.com/nix-community/comma

nix-locate can also solve that problem

A naive search is even easier. Right now, on NixOS:

   [nextos@local:~]$ python3
   The program 'python3' is not in your PATH. It is provided by several packages.
   You can make it available in an ephemeral shell by typing one of the following:
     nix-shell -p graalvmCEPackages.graalvm11-ce-full
     nix-shell -p graalvmCEPackages.graalvm17-ce-full
     nix-shell -p graalvmCEPackages.python-installable-svm-java11
     nix-shell -p graalvmCEPackages.python-installable-svm-java17
     nix-shell -p ihaskell
     nix-shell -p python311
     nix-shell -p python311Full
     nix-shell -p python312
     nix-shell -p python38
     nix-shell -p python38Full
     nix-shell -p python39
     nix-shell -p python39Full
     nix-shell -p python3Full
     nix-shell -p python3Minimal
     nix-shell -p qtile
     nix-shell -p sourcehut.python

I have zsh installed and it's not doing this. How does one get it to do this?

Are you using flakes? AFAIK `command-not-found` does not work with them. See https://github.com/NixOS/nixpkgs/issues/171054 and https://discourse.nixos.org/t/why-isnt-there-an-official-bui...

`nix-index` is a flake-compatible alternative: https://github.com/bennofs/nix-index

I am using flakes, and now I know why `command-not-found` doesn't work, thank you.

You can make it work again! It's a one-liner in configuration.nix. Solutions and alternatives were covered in detail in a recent Discourse thread: https://discourse.nixos.org/t/which-package-provides-ifconfi...

It feels like Nix is crossing some kind of developer awareness threshhold. For me as someone perusing various tech threads, this feels like the year that, among developers, Nix is going mainstream.

My main mission is to make Nix and Nix-based technology mainstream. Please reach out if you are interested, have thoughts or ideas. (plus we're hiring: https://floxdev.com/careers )

This involves improving Nix itself, as well as building tooling for teams and organizations to be successful.

It just needs a JS/TS to Nix compiler and we're good to go. lol

We just adopted nix at work so it def. feels like that from my point of view. If anyone reads this, here’s why I’ve been avoiding (and building without it) even though I wish I could use it:

- slow as hell on mac

- doesn’t work well with a project that makes use of Rust. We have weird rules for building and being able to dev with the Rust side (basically the devshell will try not to create a derivation for the rust stuff). Also every update of cargo.lock or rust-toolchain will break the nix install.

- doesn’t work well with my tools (vscode). There’s a nix env plugin but it doesn’t work with flakes. Direnv plugins exist as well but they didn’t seem to work for me.

- the shell doesn’t work without passing -i for me. Some env vars seem to break my nix

- the defaults don’t take mac into account, which is insane imo, I wrote about it here: https://mimoo.github.io/nixbyexample/

In general it feels like all the devs that are working on nix are not on mac, don’t use vscode, prolly don’t use Rust as well.

> - doesn’t work well with my tools (vscode). There’s a nix env plugin but it doesn’t work with flakes. Direnv plugins exist as well but they didn’t seem to work for me.

The direnv plugin works with flakes for me using sn envrc reading `use flake` and using the home manager package nix-direnv.

> In general it feels like all the devs that are working on nix are not on mac, don’t use vscode, prolly don’t use Rust as well.

I think this is untrue, especially for rust. I know vscode is pretty popular with nix users too.

I'd encourage asking for help on https://discourse.nixos.org. Maybe a separate thread per issue.

My current objective is to make an easy and reproducible configuration + live image installer. I want to automate the whole process. As soon as I plug in the USB stick, my preferred defaults should do their work without user input.

Although, I really don't like Nix as a language. I really wish it would have used something like GNU Guile. Searching deep for nuggets of information isn't nearly as painful as the language itself. I might adapt to it in time, but still.

Not for sure if you are working on SD card based things but this might get you half way there. We do this for loading RPIs https://github.com/Robertof/nixos-docker-sd-image-builder

Did you mention Guile because of Guix? If not, you might be interested in GNU Guix.

Can you clarify what parts you don't like. I think it's important to distinguish so I know what portions to work on (and figure out how to fix it). Is it: the language syntax itself, the idioms used in Nixpkgs, the module system, the common frameworks used to create package sets, or something else?

The language is in a weird place. The language itself is okay, but you're often trying to use the language to compute a result that meets an interface that is never defined.

There is very poor ability to reason about the 'types'/shapes of arguments or the semantics of what outputs should conform to.

It seems to require either 1) deep knowledge about the orchestration of the evaluation of the key files you interact with (e.g., what the inputs to configuration.nix will be) or 2) reading many, many examples which eventually have the hints you want.

Flakes may be a step in the right direction, but I admit I've somewhat halted my journey after finally grokking how to use nix channels and configuration.nix.

I haven't done any packaging or gotten into flakes as a unit of composable configuration. I know flakes are intended to make these things easier, but even the 'specification' for what makes a valid flake seems awfully vague.

It’s super vague. The only way I could find to understand how to create a flake (for https://mimoo.github.io/nixbyexample/) was trial and error and diving into the nix codebase.

> There is very poor ability to reason about the 'types'/shapes of arguments or the semantics

A new configuration language designed to work with Nix, Nickel https://nickel-lang.org/ looks like a step in the right direction, but I think the language design of CUE https://cuelang.org/ is the largest local maxima in the expressiveness / complexity tradeoff that we've found. Nickel is inspired by CUE, but I'm concerned that they're stumbling over the design (#1121) due to missing one of the headlining features of CUE, the U: Unification.

This ^

When I write Rust I understand exactly what structs I’m creating due to the typesystem + the excellent documentation. In Nix I’m always lost. What struct should my flake function return? What arguments can it take? Etc.

As we make flakes stable, this will improve as the docs can prioritize this information, and tooling can focus on making this more clear. On a much longer horizon, future Nix language (eg: Nickel) improvements provide a gradual typing system. On a short-term, I'm trying to provide some opinionation and a framework to describe this (https://floxdev.com/docs/custom-packages/) . The challenge is to provide enough simplicity, but not so much that one loses the very benefits we are trying to achieve.

Why does it need to be gradually typed?

If there was the equivalent to Rust doc generation that said

`fn configuration(config: Config, pkgs: Packages) -> Configuration`

"Implement this function in `/etc/nixos/configuration.nix"

And then I could click into each type and see what it's supposed to be, etc. Now, there would be some differences because Rust can't have arbitrary record types but you can imagine either forcing packages to define their inputs or just take the idea of rust doc here as a visual example. But being able to progressively drill down the types - and have the types checked by the language runtime! - would massively improve Nix's ease of use and discoverability.

> Why does it need to be gradually typed?

Because one of the things that's nice about using Nix as a consumer of Nix libraries is that for simple use cases, your code can be so simple that it truly resembles a plain configuration file.

By implementing a gradual type system, Nickel allows producers of Nickel libraries to enrich their code with helpful type annotations without making all of the code consuming those libraries verbose or cumbersome.

> being able to progressively drill down the types - and have the types checked by the language runtime! - would massively improve Nix's ease of use and discoverability.

That definitely seems to be the goal.

If you wanna get a feel for how Nickel looks today, you should try translating the exercises from the old Tour of Nix¹ on the new Nickel Playground²! The lead dev is very friendly and receives feedback warmly.

1: https://nixcloud.io/tour/?id=1

2: https://nickel-lang.org/playground

> By implementing a gradual type system, Nickel allows producers of Nickel libraries to enrich their code with helpful type annotations without making all of the code consuming those libraries verbose or cumbersome.

Those seem like different things and also on the wrong axis. The presence of types in a language does not require manually annotating everything. In fact, on the contrary, strongly typed systems have better type inference allowing fewer annotations, particularly for consumers of APIs that are typed.

> In fact, on the contrary, strongly typed systems have better type inference allowing fewer annotations, particularly for consumers of APIs that are typed.

The number of type annotations required for configuration code can't be less than zero, which is what I'd expect with Nickel.

My impression of how verbose a totally statically typed system might be for this is mostly based on what I've seen of Dhall and heard about it, which may not be fair, idk.

But the idea is that Nickel can be used for configuration code that any sysadmin would be used to writing because it might as well be JSON or YAML. This is what motivates the library/configuration distinction:

> Nickel also aims at being as interoperable with JSON as possible, and dealing with JSON values in a typed manner may be painful. For all these reasons, being untyped1 in configuration code is appealing.

> But this is not true of all code. Library code is written to be reused many times in many different settings. Although specialised in configuration, Nickel is a proper programming language, and one of its value propositions is precisely to provide abstractions to avoid repeating yourself. For reusable code, static typing sounds like the natural choice, bringing in all the usual benefits.

https://www.tweag.io/blog/2021-03-18-nickel-gradual-typing/

The Nickel docs currently cite other reasons as the main motivation for choosing gradual typing:

> For pure configuration code, consisting predominantly of data, static typing is perhaps less useful than in other kinds of application. Firstly, a configuration is a terminating program run once on fixed inputs, so basic type errors will show up at runtime. Secondly, for data validation, static types are too rigid. For example, statically checking that an expression will always evaluate to a valid port number requires very advanced machinery. On the other hand, checking this property at runtime is trivial.

https://nickel-lang.org/user-manual/correctness

https://github.com/NixOS/rfcs/pull/136 soon, you shouldn't have to care what language it uses!

What sort of time investment is required to get vaguely competent at nix?

Mostly dabble in self hosting stuff and not sure it’s worthwhile.

Been using it for about a year now. It took me a couple months of rebuilding my laptop and homelab in my spare time to really feel comfortable. I'm still no expert at the language but I understand it and have "cookbook" examples in my config repo for most things I want to do.

My advice: stick exclusively to flakes and it becomes significantly less complicated. It’s a totally different experience when you have to deal with channels and their global state.

There are tons of warnings about how flakes are experimental and this leads people who are new to nix to think that it’s going to be more straightforward to stick to the legacy method. It’s an unfortunate situation, because it’s a reasonable assumption but nothing could be further from the truth.

I'd compare it to the time investment in vim or emacs.

Getting some basic stuff done is unlikely to be difficult. (e.g. using `nix shell nixpkgs#fd` to try out the `fd` tool).

Getting a solution to a problem you have right now might be a quick copy-and-paste effort, or it might be a weekend's worth of tinkering to get even an unidiomatic solution.

Gaining familiarity with it does take a long time though. Most NixOS users will be modest about their confidence in writing Nix code (albeit, most NixOS users are also very passionate about the advantages of Nix & NixOS can be).

Jumping in and running NixOS on my desktops at work and home is how I really learned it. I'd say that aside from routine usage, several hours of doing something at least a little 'custom' with it (packaging something new, defining project environments, writing or modifying NixOS modules, writing your first override, packaging for a new language ecosystem, etc.) per week will have you feeling pretty comfortable after just a few months.

Basic stuff is easy, though, and I'd say it only takes a week or two for administering a VPS or something like that to start feeling really good. Something about the iteration loop with nixos-rebuild is satisfying and somehow very freeing. So you'll get that early payoff in a few days or maybe a couple weeks. Then you'll hit a wall and some problem will seem way harder than the experience so far. But you'll figure it out, and after a few months of occasional walls like that, real difficulties will become pretty rare. In a year, you'll be addicted. :)

For me, the Nix learning curve was challenging, but it was not a grind. It has frequent moments of payoff that start pretty early.

I'd say my main advice for keeping it enjoyable is just this: don't hesitate to engage the community if at any point you feel stumped. Nix has a truly outstanding community in terms of both technical expertise and generosity. Seeking help will accelerate your learning journey a lot, and make it feel less effortful.

That's a bit like asking "what sort of time investment is required to get vaguely competent at POSIX?"

Ask yourself what you need Nix for first.

So, I don't quite grok nix.

Am I supposed to use it instead of Ansible and Terraform? Where does it fit exactly?

Without having properly tried it either, my general impression is that the pitch goes a bit like this:

If you love Haskell and wants to make everything that has to so with packages on your system more like Haskell, Nix's got your back.

Because it's functional? I've been using Nix in production for six months and I have no desire to learn Haskell.

Nix is nothing like Haskell.

The store is built on standard Unix primitives ("worse is better"), and the core language is basically a stripped-down Javascript reskin.

Playing with Nix did increase my interest in functional programming, which was very new to me when I started using Nix.

But I've never gotten around to learning Haskell, and I don't think Nix is just for FP people.

It is a powerful abstraction that ends up being a replacement for several kinds of tools depending on your use case.

It can be a way to declaratively configure a system similar to ansible.

It can be a way to isolate runtimes (e.g. ruby v2.8 and ruby v3.0) in different projects similar to rbenv / virtualenv.

It can be a way to define an application in a self-contained way so that the target platform needn't have needed software installed beforehand similar to docker.

It can be a way to install packages/software to your system in a well-defined way similar to many package managers.

You can use as much or as little of it as you want, which I know is not a satisfying answer.

I think the most common use case is as a build system, but in this context you aren't limited to building packages, you can also build "environments". By "environments" I mean a shell with certain packages available in `$PATH`. You would use Nix like other build systems in the sense that you specify the build requirements (which other packages, libraries, etc you need to build _your_ stuff), then you specify how exactly to build your stuff. Here "stuff" could be as little as a shell script or as much a huge monorepo.

There are ways to use Nix like you would use Ansible or Terraform, but I wouldn't say that's the mainstream use case of Nix.

Nix is a package manager. But it can weave together multiple package managers so it frees you from being locked in to any one particular language, OS, architecture. The proof of this that we also ended up making a full OS (NixOS). Then there are follow-on things one can do, such as using it to manage the configuration of things and people continue to build more tools for this.

Fundamentally the core technology is for it to be a correct and safe package manager. (eg: think about the engineering tradeoffs for memory-safety vs anything-goes) Once you have that basic primitive, lots of things can be "packages": software, data, configuration, dotfiles,......

I'll take a stab and say that I believe that nix's actual power is to efficiently materialize an exact tree of files (raw or generated) on any machine. By "exact", I mean in a fully reproducible way.

That capability turns out to be really powerful because you can materialize a program, a whole Unix tree, or even a script that converges the current host to a desired state.

My use case:

I used to use Arch. I would always do a btrfs-snapshot before doing a system upgrade, then create another one after booting the new update (provided everything worked). In NixOS I don't have to because it has generations. The upgrade is entirely atomic and if it doesn't work I just use a previous generation.

Second I used to use pyenv and bunch of other tools to make sure I had the right development dependencies. Instead I just put the requirements in a shell.nix or flake.nix in the project directory, have those called with direnv. Now I can easily have python38 with clang_11 for one project and python311 with gcc48 for another.

Third the system of modules generally makes it easy to have my system setup in one place, and with the services I want. If I want zram, I just set `zramSwap.enable = true;` and that works. If I need ssh access: `services.openssh.enable = true;`, oh, but I have the firewall enabled (networking.firewall.enable = true;), that's okay because services.openssh.openFirewall defaults to true, so it handled automatically. And if I change services.openssh.ports to a better port, the firewall will be updated too.

I'm a consultant and jump around different stacks, or have to grok some new thing pretty quickly. Nix helps me with my personal dev environment. I also generally prefer functional style of doing things, declarative, immutable, etc.

Nix is so easy to share across machines (including Mac) that it ends up being pretty cool when you pair it with something like syncthing. I can jump around, get up to speed with something new, and have it everywhere without much setup now that my machines share some Nix code and data.

Mac support is pretty great with nix-darwin; I'm able to update my flake, run nix-build and my terminal/system level apps, Mac app store apps, homebrew apps, and dock shortcuts all get updated at once. My dock points to apps in the Nix store. Sure I could do that with some scripts, but Nix ends up being way more robust.

I also still use Emacs and the community emacs-overlay is pretty awesome.

But after you're up and running, Nix makes all this stuff generally seamless is my point. There have been times I can't update due to some upstream thing, but 80% of the time it's fixed for me in nixpkgs a few days later.

nixpkgs has like 80,000+ packages the last time I looked, and is growing.

I haven't had a use case to stray outside of Docker + terraform for my work related stuff, but it's amazing for sharing dev environments on a team. Flox is a new thing to check out

My repo if it helps anyone get started: https://github.com/dustinlyons/nixos-config

To me, Nix is about content addressing.

It is possible to “hash” a function so that the hash stays the same for as long as the function does the same thing (in which sense the language is similar to, e.g., Dhall), which is guaranteed because 1) function is pure, 2) the hash derives also from any other functions it may call and all the way down, and 3) everything is a function.

In the world of package management that gives you a superpower and/or curse of being able to say “I want dependency X” and mean “I want a dependency built in this exact way”. When applied thoroughly, the only inconsistencies in behavior that can arise in theory would be due to logic errors in external to Nix software or to hardware issues[0].

From provisioning deployed software runtime environments (in which sense Nix is more similar to, e.g., Docker) to automating your development machine configuration—basically anywhere you want consistency and hate surprises—in this sense Nix suffers from solving very different problems for different people, and as a result people being confused as to what is it for just by looking at articles and discussion threads about it.

[0] In practice, you might occasionally have to fight poorly behaved software that makes unwarranted assumptions about the system (can’t be installed into or look for dependencies under hash-paths), modifies itself (auto updates), etc.

Lots of people have given benefits here so i'll give some of my unique use cases that haven't been mentioned. I use it for reasons mentioned, but also;

- I work at a big org and run 10+ dev services for a mobile app team. My team uses macOS because that is what we are familiar with. We are large enough to not be able to use Docker desktop. The alternatives (Rancher Desktop, Podman) consume between 2 + 8GB before you run your app and they have TERRIBLE file system I/O. Nix gives us isolated, reproducible environments + native i/o.

- Inside our iOS project we are using Nix Shell in the build phases for things like linting, code gen etc. Ensures everyone is on the same version across developers and CI machines. Updating versions is so simple. We can also share a network attached Nix store with all the binaries to save space on CI machines.

- Once you get everyone to install it, you can add new tooling + languages for free by adding nix-env instead of bash as the she bang in your shell scripts. Need to run some ruby thing? Well you know everyone has nix installed, just do nix-shell and add the ruby version you need - its guaranteed to work everywhere. More info here: https://gist.github.com/travisbhartwell/f972aab227306edfcfea

The Nix ecosystem extends in many directions from a simple idea, which will make answers to questions like this often seem either too vague, or too expansive. But it's easy to trace the outlines of particular use cases concisely, so that's what I'll do here.

There are configuration management tools built on Nix that occupy some of the same space as Ansible, to focus on one of your examples. My favorite one is NixOS, so I'll refer to it as an example. But what I will write here about what distinguishes it from Ansible also applies to other configuration management and infrastructure-as-code tools based on Nix, such as Home Manager, Nix-Darwin, Disnix, devshell, and others.

In NixOS, IaC is achieved by using the Nix language to declare what resources must be made available on an operating system, from which Linux kernel and modules to use to what users must be present, which programs must be on the PATH, what services must be running, what ports on the firewall need to be open, and so on. This is of course very similar to Ansible, but with a different DSL.

What's different is that NixOS tries to take an approach which is, as far as possible, stateless. Whereas Ansible relies on careful management of idempotency in a series of operations to mutate the system towards the desired state, NixOS instead works by generating the desired system 'from scratch' every time you make a configuration change. To make this practical, NixOS uses some clever caching features built into Nix to avoid recreating configurations (be they config files or installed programs or anything else) that it has already created before. It does this with the same basic concepts as memoization and content-addressing.

The buzzwordy but very brief way to put this is that Ansible's approach is convergent and NixOS' approach is isomorphic. Reiterated one more way, Ansible promises that (if used correctly) it will push your configured systems in the right direction, while NixOS promises that it will create something for you whose structure is (hopefully) wholly and solely determined by the configuration you've given it.

This means, for example, that you never have to explicitly tell NixOS to remove something that you used to mention in your config file, but no longer do. Instead, when the new configuration is built, the old configuration of the system plays no role, and the software you've removed from your config file just doesn't appear in the new configuration.

I hope that illustrates the Nix approach a bit, and maybe also some of what makes it attractive for configuration management. Before I wrap up, I want to return to your last question, though:

> Where does [Nix] fit exactly?

Nix is the core of a diverse ecosystem that increasingly spans the entire devops space and the whole software lifecycle. There are tools built on Nix that integrate with or compete with popular tools from CMake to Packer to Jenkins to Docker to Bazel to Kubernetes.

At its heart, Nix itself is a DSL and software tool for managing files in a (quasi-)content-addressed store in a highly disciplined way. As a build tool and package manager, its main task is wrangling the output of the build toolchains we know and love (or not) into that big, write-once hashmap on disk. Everything else in the ecosystem is built either to leverage or enhance that basic functionality in some way.

Because it turns out that generating config files and program binaries can let you do almost anything on a Unix-like operating system, this means you can use Nix to bring determinism, rollbacks, atomicity, and other goodies to a ton of different IT and development domains. And everyone's needs and environments are different, so enterprising Nix users have grown the ecosystem in many different directions in order to integrate with their tools of choice over the years.

I'm just using it as a reproducible version of homebrew for macOS. Brew is annoying to set up again on a new install, nix is just copy my config and build it.

It's like Ansible but with a proper language, declarative instead of imperative config, a lot less leaky and brittle, a huge amount of pre-made packages and config, and can manage the entire OS. Not really the same class as Ansible actually. I used Ansible at first and it felt super hacky and broke all the time. Nix is light years ahead in reliability and power.

Don't know anything about Terraform.

Nix is not a tool, it's an ecosystem of various composable tools.

Personally I use it in complement with Ansible and systemd to deploy complex interdependent software without having to build and orchestrate containers.

Put a flake.nix in your repo and you can configure the dev environment from there: what are the dependencies, how to build your project, etc.

It’s sort of like a makefile, but it’ll always work and will setup everything for you to be able to build

regarding function docs, https://noogle.dev/ is a new search tool to that effect

I really needed this, thank you for sharing!

Author here if anyone has questions

Any good resources on how to use flakes? I'm currently experimenting with having home-manager work via a flake but I haven't come across any good HOWTOs on common operations. For example, I'd like to update the flake dependencies similar to how I used to use "nix-channel --update" to freshen the dependencies used by home-manager.

"nix flake update" should be able to do that.

This is a script I run every monday when I come into work to get the latest updates on my system [1]. It prints the flake hash when it runs, so I can also rollback if there are issues.

[1] - https://gist.github.com/J-Swift/a4dad59843f1a1f512a72308031b...

Here you go:

https://nixos.wiki/wiki/Flakes

https://tonyfinn.com/blog/nix-from-first-principles-flake-ed...

The second link appears to be down at the moment so check back later.

https://www.reddit.com/r/NixOS/comments/v2xpjm/big_list_of_f...

Maybe https://zero-to-nix.com/

I've always heard about nix operating systems. So when asked if I preferred an nix environment for development or windows, it was an easy answer. Now there is nixOS and a programming language called nix. I know this annoys me more than it should do, but it also keeps me away from the nix language and nixOS because I think the name is such a obvious stupid decision, that I'm worried what else stupid decisions they made and I don't want to invest my own time finding out.

I know people use various heuristics for quality assessment, but using the name is new to me and makes me even more anxious about naming things.

Aha, yet another tutorial since nobody can agree on anything about it seems. I can set up a nice development environment within VSCode using development containers in like 2 or 3 clicks that I can just yeet to to my GitHub for anyone to use and you don't need this kind of PhD for it.

For an average programmer or team, this stuff needs to be heavily, massively, streamlined / centralized to a single Best Practice.

Reply on news.ycombinator.com