Hacker News Reader

Comment by jbay808 | Hacker News Reader

Use Conda envs!

I feel uncomfortable with the fact that people feel a third-party solution is the best way to solve this mess. It can also get messy when packages installed with pip, pip3, conda and apt are all entangled with one another in various ways.

It’s unfortunate that it’s third party, but conda has the unquestionable advantage of being the only Python-centric packaging system that has a reasonable shared binary library story

I'm curious, do you not find wheels + manylinux reasonable? I agree that until recently, Conda definitely had that advantage, but now that you can `pip install scipy` and have that get you a working library and not try to compile things on your machine what does Conda offer beyond that?

I guess one thing Conda has that the pip ecosystem doesn't is that it supports installing non-Python shared libraries like libcurl on their own. Is that an advantage? (We absolutely could replicate that in the pip ecosystem if that was worth doing, and it's even not totally unprecedented to have non-Python binaries on PyPI.)

I think it would definitely be great if pip could install non-python dependencies. One problem right now is that many projects will tell you to just pip install xyz. You execute that, things start building, and the process fails partway with some cryptic message because you're missing an external dependency. You figure out which one, you install it, start again, and another dependency is missing. Rinse and repeat. It's definitely not a turnkey solution, and this issue trips up newcomers all the time.

With respect to versioning, I think pip should be way more strict. It should force you to freeze dependency versions before uploading to pipy, not accept "libxyz > 3.5", but require a fixed range or single version. That would make packages much less likely to break later because newer versions of their dependencies don't work the same way anymore.

Does pip allow version number dependencies? Conda is able to upgrade/downgrade packages to resolve conflicts, whereas pip just seems to check if a package exists and shrugs when there's a version conflict.

pip does handle versioned dependencies and ranges, and know enough to upgrade existing packages when needed to resolve an upgrade. Its resolver isn't currently as complete as Conda's - see https://github.com/pypa/pip/issues/988 . (On the other hand, the fact that Conda uses a real constraint solver has apparently been causing trouble at my day job when it gets stuck exploring some area of the solution space and doesn't install your packages.... so for both pip and conda you're probably better off not relying too hard on dependency resolution and specifying the versions of as many things as you can.)

... the same thing happens if you mix stuff in your usr/bin directory that isn't managed by your system package manager.

The solution is: don't mix your package environments. Use a conda environment. Just like in Linux, you'd use a container. If you wait for the Python steering committee to fix pip you'll be waiting a long time.

Conda doesn't really solve the packaging problems of python. It can make them worse if conda doesn't have a package you need.

Conda supports pip installing into a conda environment if there is no conda package.

Yes, exactly my point: then you have to potentially deal with conflicting dependencies between pip and conda packages. This happens and it's a pain to deal with.

please no.

As someone who has to look after a bunch of servers used by researchers, conda is the biggest cause of things failing for everyone.

conda installs stuff all over the place and its very easy to install conflicting stuff in the same place.

I just wish that either venv was automatic, or at least the second thing you learn in python.

Are you sure you're talking about Conda's virtual envs? Everything gets installed into the folder /envs/my-env-name.

Reply on news.ycombinator.com