Hacker News Reader

Comment by woodruffw | Hacker News Reader

This is a great summary -- epoch versioning in particular is a useful and un(der)used feature in PEP 440 versioning!

Unfortunately it's not perfectly supported. If you set up a private registry on gitlab and try to push packages with epoch versions, gitlab will normalize the ! to _ so everything breaks. (We found this out the hard way recently.)

Another Python versioning gotcha I've never see discussed is use of instructions like AVX2 in C extensions. There seems to be no way in Python versioning to say "this version is for CPUs with AVX2 instructions" so I guess public C extensions have to be compiled for the least common denominator instruction set. Does anyone know otherwise? How do other packaging systems like cargo deal with this?

> If you set up a private registry on gitlab and try to push packages with epoch versions, gitlab will normalize the ! to _ so everything breaks. (We found this out the hard way recently.)

That's unfortunate; I know Poetry's support for epochs has also been spotty (at least historically, I'm not sure about currently).

> There seems to be no way in Python versioning to say "this version is for CPUs with AVX2 instructions" so I guess public C extensions have to be compiled for the least common denominator instruction set. Does anyone know otherwise?

This is my understanding as well: x86_64 in wheel tags implies the lowest common denominator for AMD64. I think the best you can generally do here is either (1) accept the pain that comes with sdists, or (2) do runtime feature detection for things like AVX2.

> How do other packaging systems like cargo deal with this?

Cargo always builds from source, so they effectively punt on this problem.

Or simply require an AVX2-capable CPU. According to Steam survey, 90% have AVX2 support. That's not as nice as the 99.18% SSE4.2 support or the 100% SSE3 support, but for some use cases such as scientific libraries, requiring a CPU that is less than 8 years old makes sense to me. Some more helpful error message than "Illegal Instruction" would still be a good idea though. This does require runtime AVX2 detection but at least it avoids the pain of trying to compile "pristine" architecture-specific code, which seems only possible for C/C++ code that does not use any standard library function or by writing assembler directly.

Reply on news.ycombinator.com