Hacker News Reader

Comment by pdonis | Hacker News Reader

> I can imagine most of what causes problems for Gentoo is people trying to do the “right” thing of pinning an exact build backend version, which then invalidates whatever compatible build backend version they’ve picked.

This doesn't make sense. Gentoo doesn't "pick" any build backend version. They build each Python project exactly the way the project owner tells them to, using whatever build backend is specified in pyproject.toml for that project. That's why it's such a burden for them (or more precisely their users, since each Gentoo user is having to do this) if build backend versions change so often and projects don't pin a specific version but just say "use the latest"--because then they have to use what the project owner says, and the project owner says, "Update to a new build backend every week because that's how often my chosen backend, uv-build, likes to push updates".

View on HN

> if build backend versions change so often and projects don't pin a specific version but just say "use the latest"

This isn’t what’s happening: uv defaults to semver bounds, so a maintainer would have to go out of their way to introduce a troublesome backend constraint.

And note: there is no “use the latest” constraint in Python packaging. If you leave the build backend version unconstrained, the build frontend is allowed to resolve any version it sees fit, including the version it already has.

(I also know from first-hand experience that Gentoo and other distributions do not build Python packages as-is; they patch them in all kinds of ways. But that’s neither here nor there.)

> uv defaults to semver bounds

But a new patch release will be within the same semver bound, so it amounts to the same thing.

> If you leave the build backend version unconstrained, the build frontend is allowed to resolve any version it sees fit, including the version it already has.

What do build frontends actually do, though?

> But a new patch release will be within the same semver bound, so it amounts to the same thing.

Sorry, can you lay out the actual problem you’re envisioning for me? If uv-build goes from 0.13.1 to 0.13.2, then anybody with a semver bound of `uv-build>=0.12,<0.14` doesn’t need to update anything. They can keep using whatever version of the build backend they already have that satisfies that bound.

The main thing that would be troublesome is if a user pins their build backend to a specific range or version, e.g. `uv-build==0.13.1`. Then, upon bumping it, everyone who has already built 0.13.1 will need to invalidate and rebuild their build backend. But that’s why that isn’t the default.

> What do build frontends actually do, though?

uv will definitely use a compatible version. It won’t even download a copy of that that compatible version by default, since the uv binary itself is typically compatible and can invoke the build backend directly.

I’m not sure what pip and others do, but I would expect they do the same. It’s the obvious way to implement backend resolution.

> It’s the obvious way to implement backend resolution.

If a compatible version is already present, yes, a frontend can just use it if it's within the semver range specified.

But in many cases, builds are done in a fresh virtual environment that has nothing installed. In such a case, the only option is to download the latest version that's in the semver range specified. That's how the "build" frontend works, for example, unless you specify a flag to keep it from creating a fresh virtual environment.

Is that what Gentoo is doing here? I thought the whole point of the distro doing any sort of Python packaging is that they can pierce abstractions like virtual environments where it suits them.

(But also: if we’re talking virtual environments then we’re firmly in Python land, not distro land, where the correct answer is “use a wheel” instead of building it yourself.)

The whole point of Gentoo is to build everything directly from source. That means doing whatever the build source tells you to do. It also means not using a pre-built wheel, since that's not source code.

From what I can gather, Gentoo's Python build system does use a virtualenv [1]. Maybe that's not your definition of "distro land", but it appears to be Gentoo's.

It's not clear to me exactly what frontend build tool is used, whether it's one of the well known ones or something that the Gentoo developers coded themselves. However, if it's using a virtualenv, it's going to have to re-fetch the build backend and rebuild it.

[1] https://projects.gentoo.org/python/guide/basic.html

If it’s using a virtual environment, then they can pierce the abstraction by inheriting system packages, including a compatible uv-build. There’s nothing about Python packaging that requires them to rebuild the backend multiple times.

> If it’s using a virtual environment, then they can pierce the abstraction by inheriting system packages

Which defeats the purpose of doing the build in a virtualenv, which is to avoid any possible collision with or contamination by other stuff that's installed on the system by using a fresh copy of everything.

They’re already defeating the point of Python packaging in multifarious ways here. The entire reason uv and uv-build exist is because we have mature, smooth mechanisms for not rebuilding the world.

uv and uv-build are not equivalent to "Python packaging". Perhaps you would like them to be, but they're not.

I don’t know what “equivalent to” would mean here.

Speaking as someone who has written many Python packaging PEPs and who has maintained (and continues to maintain) PyPA tooling that you almost certainly use: uv and uv-build are as much a part of Python packaging as anything else that interfaces with the standards.

Reply on news.ycombinator.com