Hacker News Reader

I don't have any issues with Python's packaging ecosystem anymore, having settled comfortably into a pyenv+virtualenv+pip-tools as my "stack" after going around the block a few times.

But even so, I must recognise how awful the experience is for new users. It's taken me years to settle into this system, and it can take half a day to get someone up to speed with these tools if they haven't used them.

I also work a lot with non-developers who need to use or contribute to Python models, so that doesn't help — but I bet it would take an order of magnitude less time to get them up to speed with something like cargo. Coaching them has helped me see how user-hostile the process is to beginners.

It also doesn't help how infectious "all-in-one" Python distributions like Anaconda can be, to the point that whenever anyone has an unexpected issue one of my first reflexes is to check their PATH. The fact that Rust has a widespread default toolchain multiplexer completely solves this issue.

I appreciate that maintaining such a toolchain is work, and there is value as well in the diversity and choice of an open ecosystem. But perhaps building is one place where first-class support by the reference implementation creates a worthwhile tradeoff.

why do sometimes people say things like "and it can take half a day to get someone up to speed with these tools if they haven't used them"

half a day is like almost no time at all

half a day is just few hours, how is this a long time .. how is this any time at all

makes me doubt myself a bit, am i too mediocre to think that way

the previous line make more sense to me "It's taken me years to settle into this system" , now this is more like it

I don't think it's half a day to get proficient, it's half a day to hack something half working together so they're unblocked and can do the other stuff they want to do.

And then another half a day to get pissed off because you don't remember what settings are/are not in your venv and trying to exit and/or get back into the venv (assuming no prior experience with venv)

Fundamentally venv breaks your conceptions of what a shell is through cleverness, and that's a problem for people who are new.

Half a day to get something running that's a "one off" for you is insane. With a compiled language project, I'd download a binary. In Python, I need to reproduce the developer's setup. And I've yet to find two different Python projects where the official build instructions are compatible -- each one recommends a different environment virtualizer, a different runtime, different settings, and different C libraries that aren't part of the virtualized environment.

The thing that drives me insane about Python packaging is the degree of Stockholm Syndrome. PYTHON PACKAGING HAS KIDNAPPED YOU. YOU DON’T ACTUALLY LOVE IT.

None of Go, JavaScript, or Rust require a half a day to figure out packaging. Don’t defend Python. Push for it to change or leave it behind.

Half a day is an eternity when it is makework.

Trudging through old SO answers, bad docs, GitHub issues, overly enthusiastic blog posts by someone making a todo app, and, the coup de gras: hitting paywalls on the one tutorial that might help you.

Factor in much higher rates for more senior devs and it amounts to a lot of waste.

This whole thread is a discussion of ease vs. simplicity. Easiness is subjective, simplicity is not. We should not compare tools in terms of their ease of use. There was a famous talk on this, maybe someone can link.

I have issues precisely because of the misguided preference for virtualenvs in favor of traditional system package installation. It's obnoxious that pip now admonishes you for installing into site-packages even on a Debian system where that can't cause massive breakage. When you need isolated containers it's great. Everyone doesn't need a webdev focused, reproducible build for everyday shell life.

You really shouldn’t, though. If you use a dependency manager for some deps you should use it for all deps. Using a global/system cache would be great if dependencies were versioned and each script could specify which version is needed, but they’re not to my knowledge. And it’s all fun and games until some random install script somewhere updates a global dep and your stuff breaks and you don’t know where to even begin looking.

I’m still waiting for Nix to mature and for its UX to improve.

In my personal experience, it's absolutely necessary. Breaking changes are all over the place. I have non-dev coworkers who have built Python tools without any knowledge of package management, and it's a minefield getting it up and running.

For individualised shell usage, sure. I have global installations of common data science utilities like pandas and jupyter, or requests.

Reproducibility isn't just about deployment, it's also about coordination with colleagues.

As someone new to using Python professionally after having used it here and there over the course of 15+ years, I’ve run into exactly this problem. It’s pretty standard for a language these days to bundle the dependency manager and build tooling. Python still does this via shell infection. And since there’s 5 different ways to do it it can leave someone trying to figure out what the right vibe is in 2023 spending hours reading about the pros and cons of everything. And all that just to land back on venv+pip+requirements.txt.

Python needs a cargo. Is Poetry it? I’ve been meaning to try it…

Have you tried Rye?

https://github.com/mitsuhiko/rye

This is probably the best package manager I used for Python. It feels a lot like Cargo. It sticks to the standards of Python. No custom lock files ect. Uses prebuilt Python so you don't have to build it. Handles global installs easily.

This is very interesting indeed! A lot of the design choices fix issues I've also personally encountered. The "experimental" dissuades me from using it for real projects, but I'll be keeping an eye on it.

> Correctly installed, rye will automatically pick up the right Python without manually activating the virtualenv. That is enabled by having ~/.rye/shims at higher priority in your PATH.

I dislike this pattern. I don't want every language/tool manager in my PATH all the time. I much prefer the direnv. I hook only direnv into my shell. Then I can do what I need with an `.envrc` in each project directory.

I hate shell infection because 1) it only works in the shell, 2) it’s not stateless, and 3) depends on your working directory. I mean there’s a reason it’s called “shell infection”. Sounds like Rye specifically aims to address that issue. To each their own, I guess.

I've tried Poetry, the CLI is nice and user friendly to be sure, but it has many flaws that made me ultimately avoid it.

In my opinion, it tries to be far too "magic", hiding everything about the virtual environment and dependency management within it. This means that if anything breaks (and it does break!), I find it basically impossible to fix it without wiping everything and rebuilding the environment from scratch.

I find the documentation nice and clean to look at but somewhat clunky to navigate and very incomplete or not explicit enough — especially for beginners. For a tool that seems to want to be a "one-stop shop", you need to know a good deal already about Python packaging already to navigate it. The lack of explicitness also makes it hard to troubleshoot (as evoked above).

These are by far my biggest complaints, with the below bring mostly minor gripes.

Additionally, it seems to be targeted at library developers rather than applications (a distinction which to be fair none of the Python ecosystem makes, to the detriment of DX). This doesn't usually end up being a big issue as it's reasonably easy to "librarify" applications, but it's something else to learn. (Not every project wants wheelification as its end goal.)

The usage of pyproject.toml is nice, but the semantics of the file isn't quite as standardised as one might hope, especially when it comes to reflecting the package you're writing in the virtual environment. And the documentation around pyproject.toml is scattered and sparse. (A PEP is not documentation!)

I've complained a lot, so I should round off by saying that Poetry is a commendable project and I have high hopes for it; but as it exists now it's just too rickety for me to recommend it. But others may disagree, and by all means I'd give it a try for your next project.

My personal recommendation remains pyenv + virtualenv (or venv) + pip-tools, with simple projects just using a requirements.in files and then moving to pyproject.toml once it reaches a modest degree of maturity.

How do you build a bundle for Linux from macOS? In my experience this is only possible with Docker, and is a huge weakness of the Python ecosystem. Would love to hear solutions.

> I got so annoyed with bundler I wrote a simple shell script to provide the moral equivalent of Python venvs to gem:

This already exists: https://rvm.io/gemsets

Most people don't use it because they haven't felt the need. Bundler already makes sure it loads exactly the right versions of gems for your project, so there's no need to isolate them from each other.

I've used Python intermittently throughout my career but today it is my primary language. I've also never really had issues and have pretty much always stuck with the basics. I'm not on a very large team, though. Only 5 developers will ever touch the Python we're writing. I've only ever worked on Python on very small teams, so I wonder if that factors in?

I think complaining about Python's tooling has become like complaining about the weather in some cold and rainy country. Good for chit-chat and noding in resigned consensus.

In the meantime I cant recall any instance in the last five years (at least) where a project would fail to get going based on a simple venv and pip setup.

An exception is CUDA but I think we can agree this is not what you need to setup every day.

I wonder if that benign view due to Linux and people experiencing something very different in other OS's.