The State of Python Packaging in 2021 – Bastian Venthur's Blog | Hacker News Reader

The State of Python Packaging in 2021 – Bastian Venthur's Blog

venthur.de

43 pointsrbanffy8 comments

Python is a great language but the packaging/distribution story is not ideal. E.g. pipenv being publicly recommended but then not being updated for almost 2 years, pipenv long lock times making it unusable after your project grows large enough, poetry having pretty cryptic errors with full trace printed in output, c-extensions needing to be built, etc.

As soon as your python code needs to be used by others (library or application) things start to get messy and arcane. I will say that if all you do is deploy some python code to a server then a lot of the friction is usually avoided.

I was wondering why they are going in about pipenv? I was reading the instructions on the python packaging guide [1] (which the author is referring to as we) and by following the instructions word by word I was able to publish my package without having to touch anything other than pip and twine.

The author repeats this experience so I wonder why pipenv is out in the spotlight here?

[1] http://packaging.python.org/tutorials/packaging-projects.htm...

As someone who uses both python and Rust what I want is basically cargo but for python. So a solution that easily lets you create, run, build, publish, install python packages or software — something that has dependency managment and testing built in and is extendable with extra modules. Stuff that every python dev needs to do a hundred times a month.

Poetry comes close but it is quirky at times, and it isn't the official standard.

I understand that everything I mentioned would also be possible with the standard tools, but I don't get why the things nearly every python dev has to deal with were designed to be so convoluted, so developer-unfriendly, so unpythonic. Good tooling like this is not only about comfort and ease of entry, it is also a way to encourage best practise.

Python should have an automated testing system - something that would run the package tests on different architectures. The Perl ecosystem was really good in that - there was a testing farm comprised of personal computers people would register for it (when idle). And also the tests were run on installation - so developers were also getting much feedback on that.

This is doable, I’ve talked with various people in the Python community for years about it on and off at various conferences.

The issue is finding the money, first to pay for a little dev time, second to pay for the much more expensive infrastructure costs of running this kind of continuous build/test infrastructure.

This page, or series of pages—The State of Python Packaging in $YEAR—should have no reason to exist in the first place. But things being as they are, https://xkcd.com/1987/ is as relevant as ever. Compare what's going in the Python ecosystem with how e.g. `npm` works in NodeJS. You can hate NodeJS and npm all day long and point out a not-so-small number of problems and gotchas in the details, it's still lightyears ahead of anything Python has to offer in terms of reliability and simplicity. And it's not only Python's packaging system, it's the entire module import system that is a boat load of overcomplicated c**p. In NodeJS, when you want the contents of a module that lives at `/path/to/module.js`, you say `const x = require( '/path/to/module.js' )`. There's some amount of additional computation going on there in order to resolve relative and underspecified paths and to keep a cache of loaded modules, but other than that, the module contents are basically loaded, then `eval()`ed, that's it.

In Python, when you want to import a module from a path, you have to:

    def module_from_path( name, path ):
      import importlib
      import importlib.util
      spec                      = importlib.util.spec_from_file_location( name, path )
      module                    = importlib.util.module_from_spec( spec )
      sys.modules[ spec.name ]  = module
      spec.loader.exec_module( module )
      return importlib.import_module( name )

wat?

In what cases is this even useful or necessary?

In my 8 years of working Python, I’ve never once needed to import a module from an absolute path.

Just because doing this one task is complicated doesn’t mean that the system is bad.

Both PHP and Java have better package manager than python and arguably node. But a lot of devs don't find the language as sexy.

Reply on news.ycombinator.com