Hacker News Reader

Comment by ninetax | Hacker News Reader

Unfortunately there's a pretty annoying bug with MacOS which resulted from Apple making /nix non-writable by default. And since /nix is hard coded in all the cached packages it's not easy to fix.

This is one big thing that's preventing us from adopting nix

https://github.com/NixOS/nix/issues/2925

View on HN

It's not a bug in macOS, it's a security feature.

However, you can use synthetic firmlinks to define arbitrary root-level paths that _are_ writable.

See `man synthetic.conf` and check out https://derflounder.wordpress.com/2020/01/18/creating-root-l... for an example usage.

PS: It looks like nix has a PR open to test that solution here: https://github.com/NixOS/nix/pull/3212

However, you can use synthetic firmlinks to define arbitrary root-level paths that _are_ writable.

See `man synthetic.conf`

No, synthetic.conf only creates directories or symlinks. If it actually created firmlinks, the problem would probably be much smaller.

The problem with symlinks is that some builds use realpath. If /nix was a symlink to e.g. /usr/local/nix, this would result in /usr/local/nix being used in this builds. This leads to various build problems [1] and can result in the wrong paths ending up in binaries. If this happens on a builder, the resulting binaries are not portable between macOS + Nix machines.

So, instead the only way to get it currently working is to use synthetic.conf to create a new top-level directory and mount a APFS volume with the Nix store there.

If Apple actually allowed creating firmlinks with synthetic.conf, the Nix installation would be drastically simplified. /nix could just be a firmlink to e.g. /usr/local/nix, since realpath does not report the actual path on firmlinks.

[1] https://github.com/NixOS/nix/issues/2926#issuecomment-559989...

I stand corrected, I must have misunderstood the doc.

I'm trying to understand why macOS can't use a different path (since it's a different OS anyway, and you can't run Linux Nix binaries on macOS). From the end of the thread, it sounds like

> The main consequence of using a separate prefix for macOS is that you can't have Hydra jobsets anymore containing jobs for macOS and Linux. It would also make it harder to deploy from macOS to Linux.

i.e., if the same package builds for both Linux and macOS, you can't write a single shell script that runs on both Linux Nix and macOS Nix and calls that package? Maybe I'm misunderstanding how people deploy things, but is that a huge problem in practice?

(I don't quite follow the thing about Hydra jobsets. Aren't you building once for Linux and once for macOS anyway? Figuring out how to make the build do that seems like it would subsume any complexity of using /nix vs. /opt/nix.)

From looking at PR seems that it will utilize some extra space (for derivations that aren't really OS specific) and there might be problems wih deploying from os x to linux, but feels like they are planning to go that route ultimately.

Perhaps they hardcode full paths into whatever Nix calls its formula files (which are then, if I understand correctly, referred to by their cryptographic hashes, meaning that a file with a different path would have a different hash, and so break the whole hierarchy of formula-to-formula dependency references above it.)

You can use a different path, but then you don't have access to the binary cache, because of the mismatch between your path and the path found in the cached binaries. That's how I understand it.

Buried in that issue is a workaround, although GitHub's UI hides it from you: https://github.com/NixOS/nix/issues/2925#issuecomment-604501...

Several people in my company use the separate volume workaround and it's been totally painless.

Nix has sharp edges but this one is less of a concern than you'd think ime

Hardcoding /nix is a very big downside of Nix for me. I understand why this makes other things easier, but it's a huge compromise on flexibility.

Reply on news.ycombinator.com