Sukera

@btime reports the minimum execution time, since all increases are attributable to noise. Use @benchmark to get mean, median and maximum instead.

You probably want to check out PackageCompiler.jl (https://julialang.github.io/PackageCompiler.jl/dev/)

What kind of speed do you see now?

Would this integrate with e.g. Measurements.jl?

This is about funding for future work, not payment for achieved bonuses, it seems.

It's also weirdly editorialized, since the original title also says "Julia Computing"

Sounds similar to what stockfish (leading chess computer engine) does. In newer versions, position evaluation comes at least in part from a NN, which lead to an improvement in playing strength.

edit: similar in approach, not as a 1:1 mapping. Replace a deterministic model with a faster, slightly fuzzy one.

I'm sorry if I'm blunt, but the last year of compiler improvements have been nothing but targeted for this exact purpose? That's half of all spent time since 1.0! The issue tracker is filled to the brim with PRs and issues about making _everything_ faster. How does this constitute "refusal of a obvious step" for you?

Yes precisely - it's not the frontend that's doing the lifting here. The interactivity as you describe it depends on if the backend supports it, which not all do.

I agree that there's room for improvement here, but I also think that it won't happen in Plots.jl (and if it does, it'll depend on a backend that already provides interactivity). I think it'll be more likely to see something like this plop out of Makie.jl or something built on top of Makie.jl, as that has a lot of primitves for interactivity available already.

I'm using `!isinteractive() && main()` as my `if __name__ == '__main__'` equivalent, not sure how that's even worse?

It's not equivalent no - include doesn't introduce a namespace and neither does includet. Compiled stuff from packages (=modules with a Project.toml) is cached between runs, scripts just don't have that luxury of seperation. @from doesn't look into the files you're including and (somewhat simplified) verbatim pastes the code into your "main" file.

I don't think it's a lot of "pointless ceremony", especially since it keeps dependency management on a per project basis easy, is just a `]generate MyPkg` away and allows compiled code to be cached

If you don't want to use projects, that's fine - but please do so in a constructive manner and don't be surprised that the most common workflow (wrapping things in a package) gets more attention sooner. That just signals some disregard for other peoples' needs & wants, even if that's not intended.

The downside is that PackageCompiler (or SnoopCompile, for that matter) can only pack into the sysimage what it sees. This is usually done by "recording" which methods get called during a session and putting the compiled native code for those into the sysimage. If it's not called during setup, you'll still have dynamic compilation (as for any code that hasn't ever run).

If you don't mind me asking, how are you using julia? Are you using modules to wrap your code and functions to structure it, or are you just running one big script in top level scope?

Aside from compiler improvements, most caching related optimizations are happening on the module level, because that's where namespaces are seperated.

Note that I didn't suggest replacing structs with NamedTuples entirely - only during prototyping, while you're figuring out what you want your struct to look like. Structs most definitely will be faster.

I'm afraid I don't understand your comment - both Plots.jl as well as Makie.jl (the two most commonly used plotting packages, as far as I know) support all of those things. Makie.jl does so natively, Plots.jl does so if the backend (e.g. Plotly, PyPlot) supports interactivity (the rest is available by default). Do you mind giving an example, such that this could be improved further?

I know that there are plans to cache even more, but other than that I can basically only recommend to put distinct projects into proper projects (with a module and Project.toml). That will already cache precompiled code for that module, even between sessions. Having things in a script won't have that benefit. For experimenting with struct layouts, I've found that NamedTuples (https://docs.julialang.org/en/v1/base/base/#Core.NamedTuple) are amazing for prototyping, since they can be accessed via A.b just like structs but don't have the limitation of being const global.

The dynamism and flexibility combined with the compilation model is basically what leads down this path of recompilation, unfortunately. Since importing packages may change behaviour/invalidate some compiled method (that's what the SnoopCompile stuff in the article was about), it's nontrivial to just begin caching things left and right. You'd end up with an exponential explosion in the number of methods to cache, wasting huge amounts of disk space. That's not to say that there aren't more things that could be done, just that it's hard to do so.

The most important part about the releases since 1.0 is that compile time has been significantly reduced, the article touches on basically all ways how this has been done - may I ask when you've last tried it?

About (2), I've extensively used HTTP.jl and am pretty happy with it. I don't know about exact differences to e.g. requests, but I've found it sufficient for my uses.

Regarding (3), there's a daily CI job called "PkgEval" (which also runs before a new release is made) checking for regressions of julia vs. all registered packages, seeing if any break. This identifies misuse of internal APIs (which are allowed to break under semver) and actually breaking changes (which are then either undone or the packages are fixed). Additionally, you can [compat] bound julia itself in the Project.toml of your code.

Those two combined should mak sure you don't have to rewrite your code. All 1.x versions are backwards compatible after all.

Just a minor point - type hints only really help if you put them in struct fields. In function signatures they only limit what can be dispatched to that specific method, there's (almost always) no difference to leaving it untyped since functions are compiled for specialized types anyway.

That's not true, julia is compiled down to assembly like any regular C program would. It's just going through LLVM and most code is compiled interactively when you use the REPL, which is why you might think it's interpreted.

Python is "compiled" to python bytecode, which is interpreted by the python runtime (as in, the python runtime looks up what a function should do, does that thing, then checks the next piece of code).

Julia is compiled to first julia-IR, then LLVM IR and finally raw machine code (assembly, x86, those things) which is just run like any other compiled language. It's not interpreted.

I wouldn't call that cheating, how else would you multithread this? You basically have to chunk it up and try to find a boundary and from what I can tell, they try to find out if they're in a quoted block. If they are, they find the end of the quoted block (or EOF, which shouldn't happen => broken file) or ask the user to report the bug. This is more like failsaving, not cheating.

To each their own, but I usually try to refactor when I hit too many nested loops or branched statements - it's usually a sign of missing some abstraction or trying to be too clever.

Mind elaborating on why you think so?

One of the lecturers working on this is Grant Sanderson, of 3Blue1Brown fame. Great video series so far!

> Oh and web apps of the era were slow. Like, dog slow. However bloated and crappy the reddit app is, the old Slashdot site was slower, even on broadband.

Which reddit app are you talking about, the redesign or old.reddit.com? I ask because the old version of reddit itself certainly wasn't slow on the user side, iirc reddit moved to the new SPA because their code on the server side was nigh unmaintanable and slow because of bad practices of the time.

> Time to first paint is another concern, but that is a much more complicated issue.

That's the thing though, with static sites where JQuery is used only on updates to your data, the initial rendering is fast. Browsers are really good at rendering static content, whereas predicting what JS is going to do is really hard..

There will be more caching in the future, but until then it sounds like you'd like PackageCompiler.jl (https://github.com/JuliaLang/PackageCompiler.jl) to build DiffEq and Plots into your sysimage.

I've been using julia for about three years now and from my interactions with devs and the issues on github, caching more code has only been considered in the past year because before then, the priorities were in getting the language stable, to 1.0, and on other lower-hanging performance fruit. "Just cache more" turns out to be a harder problem than it seems at first sight, if by the very nature of the language loading a package can invalidate your caches. It has to be done with much caution, in order to reduce those invalidations as much as possible, which is why it takes time to figure it out.

The strength of using the julia REPL as a "calculator" is not in doing somewhat trivial calculations like dividing two numbers or quickly multiplying two things, and that's fine. That's not a goal of the REPL and julia is indeed overkill for those kinds of things. However, if you're mathematically inclined and sketch out a few matmuls like in the OP article, julia does have the benefit of making those kinds of interactions more bearable from a syntactic point of view. That's where it shines.

And like I've posted above, if you use certain packages frequently and restart the REPL frequently, why not use PackageCompiler to compile your hot paths into your binary, like described in the article (admittedly at the end)? That way, even the first call in a new session will be fast because it has already been compiled. On top of that, the core devs are very much aware that this is a problem and it's actively being worked on.

One way to alleviate the slowish startup is to use PackageCompiler.jl (discussed at the end of the article https://krasjet.com/rnd.wlk/julia/#packagecompiler.jl). I like that there's more than just the standard calculator operations available at the REPL, but I can see why you wouldn't use it if you don't need it and bc satisfies your needs.

2 years ago was probably pre-1.0 days. The ecosystem changed a little in the time since and the development experience is much improved (especially with Revise.jl and the in-the-works static compilation!). Depending on your application, I'd give it another go.