Hacker News Reader

Comment by scrollaway | Hacker News Reader

> Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies)

Do you have scipy/numpy/keras or cython somewhere in the deps? pipenv lock is slow, but not 20-30 mins slow unless there's a very very large download and/or a long compilation somewhere in there.

View on HN

The web app in question depends on PANDAS + numpy so that's definitely part of the toolchain. It wasn't 20-30 minutes from day one. The lock time started fast and then ballooned. Other comments here saying 2-3 minutes per lock are consistent with my general experience.

This wasn't for complex pipenv operations either. A simple command: pipenv run python main.py took progressively longer to execute.

It takes about 2 minutes (feels like 5!) on my 2016 MBP to install 102 dependencies. Doing that in Docker takes about 1.5x the time. I haven't seen it take 20-30 minutes, but 2-3 minutes is still obscenely slow in my view.

A lot of this time may be spent on downloading the dependencies to the cache. If you're doing it in docker, you likely don't have a persistent cache. I've hit this issue before. https://github.com/pypa/pipenv/issues/1785

If you configure the cache properly you might solve it, but yeah it's kinda dumb it has to do that just for locking.

I have a docker project with ~20 packages in the Pipfile, the lock step of a new `pipenv install` takes about 3 minutes.

There is no reason for Docker to be slower, it must be some kind of configuration issue. Containers are basically just processes and there is virtually no difference in execution times.

As I said in my other reply, Docker is more likely to have ephemeral storage for the cache. So every single lock it'll re-download the package. Whereas locally, you're likely to still have the packages cached.

This can make a difference of tens of minutes for some packages which have a 1 gigabyte (!!!) download.

Reply on news.ycombinator.com