Comment by PaulHoule | Hacker News Reader

Comment by PaulHoule | Hacker News Reader

Before I found poetry I worked at a place that had python projects that were too complicated for pip to download dependencies for. I had developed a tool that, for our projects, could build a set of wheels to deploy a project. I had thought about what a general solution would look like but before implementing anything I found poetry.

Poetry struck me as one of those 80% solutions that might get in the way of a 100% solution. I used it for a number of little projects and felt it was good enough but my analysis was that its dependency resolution strategy was pretty slow and that I didn't believe it was 100% correct.

For my own Python projects I've been really slumming it lately. I make venv's with pycharm and let pycharm manage my dependencies. I also have written a number of "single file Python" programs that I use on my Linux server to do things like burn DTS CD-Rs with complete CD-Text information. For those ones I just stick to the standard library. One of these days I'm going to have to deploy a real python project to a server and I'll have to think of something then.

I convinced myself that the right way to resolve dependencies in Python is to do the same thing maven does and form a graph of the dependencies of all the software versions that could possibly satisfy the constraints.

For wheel files it is practical to do because you can download the metadata in 2 or 3 http range requests out of a wheel. eggs are a problem because you are expected to download and run the python scripts inside the egg before you know anything about the egg. fortunately (1) wheels have mostly replaced eggs and (2) you can build wheels as to the needs of your organization in a private "wheelhouse" used for builds in your organization.

Since it takes a long time to fetch that information it really does have to have a local cache like maven and maybe even a metadata database. It would be nice to see python build cleaned up.

> Poetry struck me as one of those 80% solutions that might get in the way of a 100% solution. I used it for a number of little projects and felt it was good enough but my analysis was that its dependency resolution strategy was pretty slow and that I didn't believe it was 100% correct.

I think part of the problem is that almost all the solutions in the Python ecosystem (yours included if, like you say, you ever need to deploy it) are 80% solutions. It's just each one handles a different 80% of the problem. The last company I worked at also has a reasonable solution involving manual venvs and a bit of scripting on the side that worked fine until we ran into a bug that we couldn't reproduce for ages because one of our dependencies had randomly updated at some point, and we had no way of locking them. 80% fine, 20% a nightmare that I never want to touch again with a bargepole. Even looking through this thread, most people are suggesting solutions with enormous caveats to practical use.

In Poetry's defence, I think the 80% that it caters for tends to be a pretty good 80% for the majority of people, so it's still a good recommendation if you're setting up a new project that's not going too far outside the norms. But I think the 80% thing is definitely a valid criticism of it and most of the rest of the Python packaging ecosystem.

The problem is that people are concluding that "python sucks, let's use another language" every day. If you look at the replies to my comment that is what the rest of them say. That's what people concluded at that place that was having trouble building complex systems.

From that point of view it is not pip vs poetry but pip vs gopm or pip vs rubygems, or...

Maven and npm are perceived as 100% solutions in the Java and Javascript worlds respectively. Sure there are gradle and yarn but both of those are trying to be 100% solutions that are better than the other 100% solutions.

I completely agree, I say all that as someone who is happy to pragmatically work with Python, but who is also very tired of the ecosystem and wouldn't choose it for much outside of data science. Which is unpleasant to admit, because it was my first language, and I did a lot of fun stuff with it.

One of these days I'm going to have to deploy a real python project to a server and I'll have to think of something then.

That's the point at which you rewrite it in go

This was my gateway to go.

Working on a python gig targeting an airgapped windows server, different OS architecture from everything else on-hand, restrictions on building and deploying VM's, poor dev/prod workflows etc.

After a couple of months of app dev we ran into weeks of grinding through building a reproducible process (including a sneakernet step) for python dependency resolution and binary builds to get the payload dropped on the server. It was brittle and a bit overwhelming.

Go was "big enough" at the time to be a reasonable next step and after a few weeks of porting we had a cross-compilation process up and were done.

Go actually _shits me to tears_ as a language compared to python, rust, or even Lua, but as a back-end dev it's still where I do most of my work as I'm nearly guaranteed to be able to i) predictably solve something in a way i can share it with a team and ii) get it deployed without too much pain.

Reply on news.ycombinator.com