Hacker News Reader

Comment by sandGorgon | Hacker News Reader

So I have been considering this. does conda track pypi or does it lag it? I have been concerned about moving over my requirements.text for a webapp with lots of dependencies

View on HN

It slightly lags, but you can include pip requirements in an environment.yml file, and they install normally.

I really only use conda for the non-python bits of our stack: numpy/scipy/pandas etc - packages that are a pain to install on Heroku.

It's also pretty straightforward to set up your own Conda package tree. Nice for packaging your app for deployment or making sure you have very precise dependencies.

http://conda.pydata.org/docs/custom-channels.html

I think deployment is a solved problem with docker. Its libraries like blas,etc that are a huge pain. I'm not sure why static linked bumpy is not possible - even anaconda could not achieve it.

If you've ever tried to dive into the NumPy build process you'd see why. It's unbelievably complicated... not that they really could do it better given that they are compiling about a billion scientific libraries and support alternatives and optimizations (like MKL).

Yes - unfortunately I have and I failed miserably. These days I'm trying to see if there's a docker build that can build a great numpy (with all optimizations). Interestingly there are even docker images to call cuda APIs from python.

They are a pain to install on a desktop as well!

Especially with all the blas linking. Was there anything special you had to do or was it simply conda install numpy-blas or something like that ?

Fro Heroku we created/modified a custom buildpack (https://github.com/joshowen/conda-buildpack), and use that with the multibuildpack.

We use conda-env (https://github.com/conda/conda-env) which makes it really simple to manage environments locally and in remote environments.

We have to use a mix of pypi and conda since quite a few of our dependencies are not in conda. We have a script which checks conda first, then falls back to pypi, all from one requirements.txt

Why not conda all the way? Anything of concern?

Incidentally is the buildscript of anaconda itself opensource? Couldn't find it anywhere.

Not sure if this is what you're looking for: https://github.com/conda/conda

Any chance you can share that script? I'm looking for something similar, since I too am using both conda and pypi for dependencies.

https://gist.github.com/dwiel/7a2c0163b802e1cea0f6

Instead of requirements.txt we use py.prereqs, but otherwise this script should get you close.

Reply on news.ycombinator.com