Hacker News Reader

Comment by ihuman | Hacker News Reader

Is there a more verbose explanation about why they’re moving away from compiling packages from source? All the explanations I’ve seen make it sound like that’s the obvious direction it should go, but it’s not obvious to me.

View on HN

I got into a minor argument with the maintainers about it, and the response I got was that they thought it was an unreasonable support burden: https://discourse.brew.sh/t/maintaining-a-tap-to-provide-opt.... Not something I could get them to provide more details about, however. Not really a fan of the response I got :/

The actual response:

> We’re not willing to ship things we don’t support, sorry. I do not consider it acceptable to ship software that we are unable to test and we know will break. Even if I did consider that acceptable I’d be unwilling to accept the additional support burden that the maintainers have to take on from people submitting issues when they are clearly told not to (something that still happens often).

Clarification beyond that: fixing avoidable user issues is not what most of the maintainers want to spend most of their (volunteer, mostly evening and weekend) time on. Even immediately closing issues as WONTFIX takes more time to deal with than not getting an issue at all.

You may disagree with this (clearly some people do) but ultimately you cannot expect a volunteer run project to support more than they are willing to do. If we had people stepping up and saying "I will deal with all the support requests for this!" that'd be different but mostly we have people who want us to do the work for them.

What happens to all of my packages I have installed with options? When I upgrade them next they will suddenly be stripped down? ffmpeg is a huge one, as the default set of options excludes many things that most people using ffmpeg need by default, but homebrew can't ship as a binary for legal reasons.

Yes, they get reinstalled without the options.

As a user, you can always ask for the package to be compiled from source. This is also what will happen if you ask for non-standard options. The advantage of using binary packages is time: I have a Homebrew formula that takes 40 minutes to build from source, but maybe ten seconds to install from a binary package.

Options have been deprecated, and I haven’t been able to find anything that replaces them that the maintainers are willing to discuss.

Options have been removed in the main Homebrew/homebrew-core tap but are not deprecated for user taps.

Maybe ask the question the other way around. Why should they keep compiling everything from source? What are the benefits?

You get native binaries, can choose optional features and dependencies and you can easily apply your own choice of patches. That's my view as a gentoo user anyway; I don't know how much of that homebrew actually supports.

Homebrew used to support optional features, but is now moving away from this to become a binary package manager.

What does “native binary” mean?

Compiled locally, so (posssibly) optimized for better performance than a generic binary compiled for your architecture.

Compilers like GCC can produce code specifically for your CPU such that it uses all of the special features you might have like SSE, AVX etc. When binaries are made for general use they have to be compiled with a generic feature set so might not make best use of your CPU.

So can a cross compiler, no?

If you know the exact flags for your CPU then you can compile for it on any other CPU using a cross compiler line GCC, yes. But that doesn't help because in practice you'll just find one binary per architecture, not per CPU.

At least it should no longer take a night to compile some GUI app like back in the days.

Well, for one, as a longtime brew user, changing to binary packages alters the trust model.

Binaries are built on CI infra with logs that are publicly available and with all the source code doing the builds publicly available. Effectively the same process as local, from source builds. The bottle DSL ensures that the SHA binaries are not tampered with between when they were built (and the source code sha checked) and when they are delivered.

I know this is still a different trust model, but, IMO, not too different.

I know reproducible builds are on the radar for Homebrew in the future, and that should ease some of your concerns too.

For a good thread on Homebrew security checkout https://twitter.com/c_pellegrino/status/1093195802871246848

Hey! Thanks for the reply. I’ll definitely check that link out. It’s really nice to know more about the process behind delivering binaries.

Compare to e.g. conda, where you get binary packages and no option to build from source at all. This basically means you get support for x86 and very limited support for PPC. Need ARM? Nope, no can do. Or any other slightly exotic architecture.

It also means, though this is only a problem with third party channels, that it's possible to get into a situation where you can't even build from source at all. I was working with one of our custom packages, where for the life of me I could not figure out how to get the build to work. Turns out it had been hardcoded to run on a very specific machine, and assumed access to a very specific network file system, and I wasted about a week of my life trying to untangle those assumptions before giving up. If that software had been designed from the beginning with the idea that it ought to be possible to build from source from anywhere, we'd presumably be in a different situation now.

Conda has support for armv7 via berryconda.

Interesting, good to know. I'm guessing third party channel support is probably close to nil though, so for my use case it probably wouldn't get me very far.

In short:

- it's faster for end-users

- it's more reliable (which both makes the user experience better for users and reduces the support burden for our volunteer maintainers)

Reply on news.ycombinator.com