Hacker News Reader

Comment by apitman | Hacker News Reader

A while back I was trying to understand why Homebrew requires pre-built executables to be installed into /home/linuxbrew. I asked about it here[0]. This requirement basically makes it impossible to use homebrew to quickly install programs on systems where you don't have root, or at least have homebrew already configured (not sure if that would solve it but I assume so).

They pointed me to an example program that would break if not run this way: Facebook's Watchman[1].

It bizarrely (to me) has hard coded paths compiled into it, which force you to run it from specific directories.

Would love to understand what's going on here and why you would ever make software work this way. I feel I'm missing a fairly obvious Chesterton's Fence.

[0]: https://github.com/orgs/Homebrew/discussions/5371

[1]: https://facebook.github.io/watchman/docs/install#prebuilt-bi...

View on HN

The short (but possibly not satisfying) answer is that Homebrew's relocation of packages (including binary relocation) is best effort, in part because of the myriad ways in which packages can embed absolute (or incorrect relative) paths and other state in their build products. macOS bottles are generally more relocatable (in part because of a lot of scar tissue around binary relocation), but it's a general problem with build system quality, build complexities, and - reasonably - disinterested upstreams.

In the case of Watchman, I have to assume that internal use is the most supported use case, and uniformity of deployment is desirable across the fleet there, and so, configurability wasn't as big of a concern?

My short experience with Watchman (a few years ago) indicates this. It’s pretty clearly only technically open-source, without much regard at all paid to third parties actually using it.

I’ve had a few build pipelines break over the years because of a watchman dependency. IIRC it was usually an issue with an npm library depending on watchman but downloading a binary that was incompatible with the architecture or implemented the wrong syscalls for the operating system.

That makes sense. The weird part to me is that Homebrew would limit their approach and eliminate an entire class of use cases to accommodate programs that work this way. There has to be more to it.

I don't think that's accurate—homebrew specifically says that it only uses the .linuxbrew directory when a formula contains a hardcoded path (which it scans for), and only if you choose not to install it from source.

So, based on the responses from the maintainers, for the .linuxbrew directory to be used, you have to satisfy 2 conditions:

1. you have to be installing one of the ~10% of formula that isn't trivially relocatable.

2. you have to be using a precompiled binary (which it seems like homebrew is smart enough to not do if condition #1 fails and you're not using sudo)

This a not a unique problem to homebrew. Any pre-built binaries potentially shares this problem unless the build system the software use is intentionally written to avoid this problems.

Any package managers that is designed to not hard code the prefix, ie you can choose where the binaries go into, needs to handle this problem and have their own ways to deal with it. Conda for example has a long string of …placehold_placehold_… to facilitate editing the hardcoded path…

Source distribution is more robust against this problem comparing to binary distribution. (But sometimes the authors of the software did not package them well and would have hardcoded some paths somewhere.)

That’s why when you change the homebrew prefix, they will built from source instead, and it (using a different prefix) might not work.

Reply on news.ycombinator.com