Hacker News Reader

Comment by vchuravy | Hacker News Reader

My point is if you set JULIA_CPU_TARGET during the docker build process, you will get relocatable binaries that are multi-versioned and will work on other micro-architecture? It's not just for PackageCompiler, but also for Julia's native code cache.

View on HN

It worked! I was able to drop the Windows install on a standard GitHub Actions worker from 1 hour to 27 minutes. Here's what worked:

    ARG JULIA_CPU_TARGET="generic;skylake-avx512,clone_all;cascadelake,clone_all;icelake-server,clone_all;sapphirerapids,clone_all;znver4,clone_all;znver2,clone_all"
    ARG JULIA_PROJECT=[...]
    ENV JULIA_PROJECT=[...]
    RUN julia -e "using Pkg; Pkg.activate(\"[...]\"); Pkg.instantiate(); Pkg.precompile();"

What I got wrong the first time: I failed to actually export JULIA_CPU_TARGET so it would take effect in the "Pkg.precompile()" command. In reality, I hadn't correctly tested with that environment variable set at all. I was only correctly setting it when running PackageCompiler.

Thank you so much for this! It's too late for me to edit my original post, but cutting the install time in half is a major win for me. Now it only needs to precompile, not also compile a sysimage.

... Actually, this only worked for Linux. My Windows container is back to precompiling every time, again, and Windows was the slow one that I wanted to fix in the first place. I had to revert this. Back to the drawing board. I wish Julia would print some diagnostics about why it decided to precompile again.

I think I finally nailed it? I needed to update JULIA_DEPOT_PATH since I was relocating the cache files. I knew that and was already doing it, but I didn't do it right: I needed a trailing semicolon so it would have an empty entry at the end! The documentation talks about this but I didn't understand it at the time: https://docs.julialang.org/en/v1/manual/environment-variable...

That was the very first thing I tried, and I couldn't get it to work, but I'm sure I am doing something wrong. Everything seemed great at build time, and then it just precompiles again at runtime, without anything saying why it decided to do that. I'll give it another shot if you say it should be working. The PackageCompiler step is the longest part; if that can be removed, it'll make a big difference. I'd rather be wrong and have this working than the other way around :) I'll report back with what I find.

Reply on news.ycombinator.com