Hmm, this is solvable in theory by defining a platform tag for musl such that authors of C extensions could upload musl wheels, right?
I see some discussion at https://github.com/pypa/manylinux/issues/37 but I don't totally follow why it died off.
I see some discussion at https://github.com/pypa/manylinux/issues/37 but I don't totally follow why it died off.
(I have contributed a tiny bit to the manylinux stuff, so I'm curious where the problems are, but I'm definitely not active in any sense... it does seem like a couple of these issues boil down to "there should be more people spending time on it," sadly.)
This gets pretty difficult when you are required to do it in Docker in order to produce manylinux-compatible wheels. Or at least that is my understanding.
For some of the arches I need to support I just do it on real hardware - but of course the quay.io manylinux images aren't available for those arches. It is a half baked solution.
> Is it that PyPI won't actually accept those wheels and so you'd have to self-host?
Yeah, that's basically what it boils down to. The platform tags are underspecified - especially for ARM, though this is hardly a python specific problem since few people seem to really understand what the differences are in ARM ABIs - but PyPI accepts a subset of even those, which is limiting.
The other difficulty is that the packaging infrastructure for impure wheels that aren't strictly cpython extensions is basically nonexistent. The packaging procedure for a module that imports a .so via ctypes, for example, is basically "good luck".
To digress slightly, another problem is that source distributions for such packages so frequently throw some obscure python stack trace or a "lol no gcc" stack trace when you try to install them in pip. Which to me is an unacceptable way to communicate a problem. Pip's apparent worldview that tracebacks are an okay way to communicate errors is so hostile to end users, and I see that same attitude present throughout the Python ecosystem. I mean the whole concept of venv's as standard procedure is a good indicator the packaging system is fubar. Sorry for the rant.
For comparison, with Ruby you use RVM or Rbenv plus a Gemfile with Bundler and you’re done. There’s no debate to speak of.
The seemingly equivalent nightmare that I see in JavaScript inspires me to go use Ember if only because its creator was instrumental in forming the modern Ruby tooling ecosystem, and I hear that ember is equally unlikely to make me want to stab my forehead with a fork in frustration.
Judging by what I’ve read here and experienced firsthand with the state of python tooling, this seems like it can be a worthwhile trade off
The reason this has never become a problem in Ruby world is because Ruby community is centered around Rails, of which a requirement to deploy on non-x86_64/ARM64 is pretty much non-existent. Ruby itself doesn't even support non-glibc (e.g. compiling Ruby on musl requires a patch).
https://github.com/rubyjs/libv8#do-i-get-a-binary would like a word about that. You get source or binary depending on your platform / version.
It's entirely possible that the Alpine user base isn't as big/important as I would think but this seems like something that needs to be solved.