Hacker News Reader

Comment by YetAnotherNick | Hacker News Reader

I wish there is some package manager in middle of conda and pip. Conda is too strict and often get stuck in SAT solving. pip doesn't even ask when reinstalling a version currently being used.

Edit: Typo: reinstalling a version of package currently being used

View on HN

My ask would be to get rid of the need for conda all together.

Conda obviously offers a lot of value in sharing hairy compiled packages, but it does not play well with anything else. None of available the tooling really works with both conda and pip. It fragments the already lousy packaging story.

other than an occasional pkg conflict, we've found conda and pip to work pretty well together (defining pip dependencies in the conda env) -- across a lot of different python envs

Try poetry. It wraps pip and fixes a lot of its issues

Seconding Poetry. IMO it should have been the standard package manager - it just works (TM)

Although it consumed 70GB of RAM before I killed it, when I tried to use it to `poetry install` stable-diffusion.

we find mamba solves deps solving for conda (we fail to do GPU dependencies without it), and I think it's getting integrated

my main thing w/ conda is it's bananas figuring out how to make a new recipe, which is pretty surprising

Agreed, I've found packaging for conda to be so much harder than packaging for pip

Conda sucks because it always wants things its own way, and introduces yet another site-packages and environment I need to track. Absolutely hate Apple for requiring it for their metal and M1 support.

> pip doesn't even ask when reinstalling a version currently being used.

Just as an explanation: a "version" in Python packaging can come from one of many potential distributions, including a local distribution (such as a path on disk) that might different from a canonical released distribution on PyPI. Having `pip install ...` always re-install based on its candidate selection rules is generally good (IMO), since an explicit run of `pip install` implies user intent to search for a potentially new or changed distribution.

I meant this for dependency not the package I am installing.

I'm not sure I understand what you mean -- `pip` should not be reinstalling transitive dependencies. If you install A and B and both depend on C, C should only be installed once.

I think they're referring to requirements files. I've seen the same behavior - on day 1, pip installs packages A and B, then on day 2 when someone else modified the requirements file it installs C and reinstalls B even though B hasn't changed.

This one I've seen when the dependency doesn't specify an exact version and you included "--upgrade".

There's a second case that I think was fixed in pip 21 or 22, where two transitive dependencies overlap - A and B depend on C, but with different version ranges. If A allows a newer version of C than B allows, C can get installed twice.

Try using mamba (https://github.com/mamba-org/mamba)

We ran into many unsolvable or 30m+ solvable envs with conda that mamba handled quickly.

The underlying solver can be used with conda directly as well, but I have not done that (https://www.anaconda.com/blog/a-faster-conda-for-a-growing-c...)

Sounds like you want pipenv, but that might be too close to conda

Reply on news.ycombinator.com