Hacker News Reader

Comment by systemvoltage | Hacker News Reader

My experience has been completely the opposite. Julia has been frustrating to work with. Most of the time I’m just waiting for precompilation of the packages.

It’s unbelievable that the Julia core devs still haven’t addressed it. It is not fun by any means. Not even tolerable.

View on HN

I work with julia every day and really don't spend much time waiting for things like compilation or precompilation.

> It’s unbelievable that the Julia core devs still haven’t addressed it. It is not fun by any means. Not even tolerable.

While I never really thought this was a big deal, the Julia devs certainly do, and I'm a little upset on their behalf that you'd say this. An amazing amount of developer time and effort has been going into reducing loading times and compiler latency lately. The currently release, 1.5.2 is leagues faster at package loading than the previous version and the upcoming 1.6 release candidate is even faster.

They recently implemented multi-threaded precomilation and have been going on a method invalidation crusade doing largely thankless work to deal with exactly this sort of whining.

I applaud Julia developers for their work. They just haven't focused on the UX of using Julia on a daily basis. I have been forced to use Julia as a maintainer (not author) and I feel totally exhausted, defeated and uninspired after working on Julia code.

Just being honest and voicing my opinions. If it hurts the people that worked on it for years, I am sorry.

Have you asked on https://discourse.julialang.org/ already? Maybe, hopefully, you can find help there to feel less exhausted.

-- It's not an opinion when you e.g. say that precompilation has not been addressed when it is being worked on (with much cost, I'd suppose). Same for haven't focused on the UX [snip]. In case you aren't on the julialang discourse I really encourage you to register and ask.

Thanks, I haven’t reached out but mostly seek help on SO.

SO is fine. On discourse (besides asking questions like on SO) there is also space to e.g. discuss why one feels exhausted, defeated and uninspired after working with Julia. This shouldn't be! And there is certainly a way to "fix it" :)

I am not sure faster compilation is the way to go. Cutting 30 sec to 15 helps little and the acceleration of compilation can't catch up with the growth of packages in the long run. Also, multi-threaded precompilation sounds a bad idea on managed clusters where resource usage should be controlled. I wonder how much Julia devs have done on caching precompiled binaries. That, though non-trivial to implement properly, is the ultimate solution to slow compilation.

The manual says “Large modules can take several seconds to load because executing all of the statements in a module often involves compiling a large amount of code. Julia creates precompiled caches of the module to reduce this time.”

https://docs.julialang.org/en/v1/manual/modules/#Module-init...

Just to clarify, those precompiled caches are (currently) not of binaries. At the very least, LLVM still needs to generate code after loading, and often everything up to inference still needs to be run as well.

Julia 1.6 involved a lot of work reducing invalidations, where old methods are invalidated by new definitions and thus forced to recompile. This has the obvious benefit of reducing the total amount of compilation needed, but the secondary benefit of making compiling to binaries more profitable; it's less likely that work will be wasted when someone loads another package.

This sounds great in theory, but in practice julia-1.5.2 still takes 10 sec to start when I use a medium-sized package including its dependencies (16 compiled files in total). If julia is loading cached files, I would expect something below 1 sec.

You can manually precompile a system image with libraries you find yourself using often, like Plots.jl. The usability of it stinks, but... you can do it, and it speeds common things up.

https://julialang.github.io/PackageCompiler.jl/dev/examples/...

Its usability is being integrated into the package manager so that it automatically gives warnings and triggers when you need to redo the sysimage. Not in v1.6, but there's been a lot of discussions for doing it shortly after. That is something I'm excited for.

Thanks for the tip. This does look promising.

We recently acquired new hardware and plan to look into it. Invalidations can get in the way, but there are some things that can be done.

That's what pushed me away in 2015. Someone said it was fixed in... 2018 I think, but I tried and it wasn't. It still isn't? Ouch.

If it comes to pushing binaries into the package manager, that's what it comes to, but waiting for a minute for the graphing library to load just isn't OK in this day and age.

I just tried it, and it took 13 seconds to load and plot some random numbers. It could be better but it’s an acceptable trade off for my usage. Not as good as Python in this regard, but much better than Matlab.

It's approximately halved in Julia v1.6 BTW. It's not perfect but continually getting better.

Is 13 seconds a build cache hit or miss?

It’s a precompilation hit, but it requires some compilation every time. Thinking about Julia like you might think about incremental C++ builds isn’t exactly right. A somewhat useful analogy though is that Julia is kind of like C++ template code. This is why it is so flexible, but also why compilation isn’t very cacheable. This is also why it is tricky (but possible) to do ahead of time Julia builds.

I don't think it's fair to say that it has not been addressed.

See this recent blog post for instance: https://julialang.org/blog/2020/08/invalidations/

Try to use the REPL more instead of re-running your program from scratch. I just do `include("blah.jl")` then re-run my testcase.

Ah, the statefulness foot-gun. It'll solve all your problems... if your problem is having feet!

Ok, that's a little mean, there's nothing quite as convenient and simple as a REPL or notebook for ad-hoc caching, but sometimes you need the opposite and the language really needs to support that.

I often chain unix tools, domain-specific binary executables and python scripts in shell. Julia feels very slow for this type of workflow. A few years ago there were discussions about whether Julia could replace Python. No way just due to Julia's long startup time.

Ditto.

I can't wait for Julia to murder Python but before it can do that it needs to push all delays out of the interactive parts (to include both REPL and cold launch) and into the package management parts, where people will tolerate delays. Anaconda decided to stick a full SAT solver into the package management and it's every bit the shitshow that you'd expect, frequently taking 30 minutes to solve and failing to converge, but people tolerate it because it's in the package management.

I have been using anaconda much less often due to its long resolving time... That said, I fully agree with you that Julia should cache precompiled binaries. Lacking this key functionality is a misstep from the very beginning.

That's my least favorite thing about R - it just looooves to save state unless you configure it otherwise. This is also why I've had trouble adopting Julia in the past - I've adopted the habit of constantly resetting my workspace or working from scripts to avoid exactly this issue, so my habits are somewhat Julia-hostile.

I'm giving it another go though, since it seems like it could be really useful for algorithm-adjacent research.

It's a bit annoying but you should only have to precompile once and it doesn't take that long

That's not true though. It's everytime I want to do anything with Julia, I need to wait upto a minute or two. Gadfly takes ages to pre-compile.

Closed your terminal session? You're outta luck. Wait 2 mins please. Launch a notebook and want to start working on something? Wait 40 seconds. Wanna check something quick in Julia REPL session? Wait 7 seconds. Let me run this Julia script, wait 55 seconds.

I feel like the entire community should stop doing what they're doing and fix this issue. I am afraid, it is difficult to fix otherwise it would have been fixed by now.

This is also roughly my experience (though I haven’t tried 1.6 yet) and it really a shame - I much prefer Julia’s syntax to R’s (my primary language) and would love to switch for statistics work.

The problem is perhaps that I largely do fairly simple statistics and plotting, with scripts that in R usually take seconds rather than minutes to run, and the disadvantages of the requirement to compile the code overcomes the compiled speed advantage.

I have the same use case as you describe and it's not unreasonable to conclude that right now Julia simply does not offer anything that R does not. It seems most Julia users are more in fields like numerical computing or scientific simulations and such, where it does offer clear advantages. But if you just want to run some of the statistical tests you learn in undergrad or want to explore some data R is just a better choice right now.

You can speed this up using a custom system image, but the usability of it kinda sucks.

https://julialang.github.io/PackageCompiler.jl/dev/examples/...

Then they should have the package manager do it. Or the package maintainer.

The current release candidate for 1.6 does exactly that. In parallel.

Sweet!

Reply on news.ycombinator.com