Hacker News Reader

Comment by denom | Hacker News Reader

In my particular use-case, I'm using a set of local dev tools hosted as a homebrew tap.

The build looks up the github tar.gz release for each tag and commits the sha256sum of that file to the formula

What's odd is that all the _historical_ tags have broken release shasums. Does this mean the entire set of zip/tar.gz archives has been rebuilt? That could be a problem, as perhaps you cannot easily back out of this change...

View on HN

They never really stored them, they were always generated by some code (maybe with a cache layer in front). The code changed in a way that changed the bytes in the tar.gz without affecting their contents-when-extracted.

The trick here is that a Github release is in essence simply a tag of a specific commit. There is no need to build archives in advance, as they can be dynamically generated from the git repo.

However, if you change the compression algorithm used to generate the archive, it'll result in a different checksum! The content is the same, but the archive is not.

> Does this mean the entire set of zip/tar.gz archives has been rebuilt?

They are probably generated on-demand (and cached) from the Git repository, not prebuilt.

I think the zip/tar.gz archives are being created on the fly when you download them, probably with a caching layer in front.

Pretty bizarre this ever was stable in the first place.

Unfortunately for this kind of service you need to actively fiddle with the bytes to prevent people from relying on an implementation detail like this and prevent them from digging you into a too big to fail api stability hole.

Alternatively, they could extract the compression code and maintain it for repo tags created before the git algo update release date.

Isn’t that the only humane course given all that depends on this?

That's my thought as well. They could also potentially retroactively generate the source tarballs using the old method for every possible repository/tag on Github, store it, and serve that, and then only generate it on-demand for new tags, but I doubt they'll do that. They might though, given this is what led to the problem in the first place (ie; the on-demand generation vs generating on push+storing).

That seems wasteful. Many projects do not actively advertise the GitHub tag downloads, and instead have their own stored and stable tarballs (or other distributions). And I suppose many users of those auto-generated downloads don’t care about their checksums.

The solution is simple: for a monthly payment of $1 per gigabyte the downloads are stable. Otherwise they are not.

Who should pay for this? The developer of the project might not care about the stability, but the maintainers of the various Linux distros might. Should the developer pay from their own pocket to make the Linux folks happy? Should there be some pool of all the Linux distros to collect the fees and pay for these projects?

Reply on news.ycombinator.com