Hacker News Reader

Comment by heywhatupboys | Hacker News Reader

Python's package ecosystem is probably the worst in the world

Worse than Javascript's 5+ different types of mostly incompatible module formats, transpilers for different version targets, megalithic node_modules directories and the community's love of importing new packages for one-line functions that can disappear and take down the internet?

How is `node_modules` different from `/path/to/venv/lib/python3.X/site-packages` ?

Are PyPI, cargo, Maven, etc... all immune to supply chain attacks?

One major difference is a venv doesn't allow nested dependencies. With npm, you might install version 2.3 of a package, only to find that some other package has installed version 1.4 under itself.

Python also has "wheels", which are pre-compiled distributions, rather than full-source distributions. These are often smaller and install faster.

> One major difference is a venv doesn't allow nested dependencies. With npm, you might install version 2.3 of a package, only to find that some other package has installed version 1.4 under itself.

a feature not a bug. Anyone who ahs built a sufficiently large project and had to abandon some dependencies know this pain

Yes. Which is quite impressive.

As much as we all have criticized the npm ecosystem, after seeing what exists in the python world I actually came round to it and consider it very good.

Javascript is a larger technical mess once you pull it open and look inside.

But for day to day use it's much better than Python. They've done a fantastic job overall allowing you not to need to worry about all the different formats and transpilers and whatnot.

In Python at least you "only" have to figure out venv, pip and conda and you're mostly set. In C/C++ world adding a dependency is such a pain that most projects have less than five dependencies. Most of the time it's simpler to just quickly write and debug a worse version yourself than to figure out how to find and integrate a library. Until you reach sufficient scale, at which point the sane thing seems to be to make your own build system instead

>In Python at least you "only" have to figure out venv, pip and conda and you're mostly set. In C/C++ world adding a dependency is such a pain that most projects have less than five dependencies. Most of the time it's simpler to just quickly write and debug a worse version yourself than to figure out how to find and integrate a library.

That's from the developer's perspective, not the user's. The user of a C++ project doesn't have to care about any of that; they just run the binary, or ./configure && make to build from source. The user of a Python project has to struggle through worst case days of pain depending on how many dependencies are randomly broken.

If your user is interacting with the package ecosystem, you already failed multiple steps. Which python projects are routinely doing, but that's a cultural issue about packaging.

For "users" that have to touch the code: well, those get the full developer's perspective. And I've lost more hours than I would like to admit trying to get C++ projects to build. A project has to be really small or packaged really well to make it as simple as ./configure && make

Can you elaborate what this system lacks or what are the key properties that make it „the worst“? I know its a mess, but I cant pinpoint the reason why.

Is the fact that there are multiple package managers for the same package fornat/spec the reason why people say it's a mess?

Genuine question: How is PyPI different from NPM, Maven, crates.io, etc..?

Why "Python packaging is a mess" and not the others?

That so many things doesn't run on the latest stable version is one of them. Python 3.11 was published in all the way back in October, and it's still not stable and well supported.

The non-hierarchical, not-reused, reinvented package convention. The most popular package repository being incredibly hard to find one package you want among a sea of crap. The lack of a single standard solution to the most common problems. Frequent backwards compatibility issues.

It's definitely not the worst. But it's hilarious how bad it is considering how popular it is and how long it's been around

try ruby.

What's wrong with bundler / gem?

System Ruby basically breaks "gem install", right? However once you install your own Ruby as a user, it's easy to use.

Interestingly, Ruby was the first language to improve on Perl's CPAN. Maven 2 was inspired by it.

For their age, bundler/gem do quite well.

Yes, it is way better which is why many layer package managers, e.g. cargo and composer, took the ideas from Ruby to improve on. Ruby has issues but the package management is not one of them.

Reply on news.ycombinator.com