Hacker News Reader

Comment by anon389r58r58 | Hacker News Reader

Almost feels like a fallacy of Julia at this point, on the one hand Julia really needs a stable, high-performance AD-engine, but on the other hand it seems to be fairly easy to get a minimal AD-package off the ground.

And so the perennial cycle continues and another Julia AD-package emerges, and ignores all/most previous work in order to claim novelty.

Without a claim for a complete list: ReverseDiff.jl, ForwardDiff.jl, Zygote.jl, Enzyme.jl, Tangent.jl, Diffractor.jl, and many more whose name has disappeared in the short history of Julia...

View on HN

I do not think this is meant to be a “real” library, but a didactic exercise inspired by Andrej Kaparthy‘s Python implementation.

This is a didactic exercise. Julia is fantastic, but lacks funding to develop a differentiable programming ecosystem that can compete with Torch or Jax. These two have corporate juggernauts backing them. Still, it is quite remarkable how far Julia has got with few resources.

Having an alternative to Python would benefit the ML ecosystem, which is too much of a monoculture right now. Julia has some really interesting statistics, probabilistic programming and physics-informed ML packages.

I think you are asking an ill-posed question in parts. Julia has a lot of great things, and needs to continue evolving to find an even better fit amongst the many programming languages available today and sustain itself long-term.

Emulating Python's ML ecosystem is not going to be a viable strategy. The investment into the Python-based standard is just too large.

What I could see happening though is that the continuous evolution of the ML ecosystem will further abstract components of the software stack down to an MLIR/LLVM abstraction level at which point something like Julia could also use this componentry. Sort of a continuum of components, where the "frontend" language and associated programming style is the remaining choice for the user to make.

That's what Reactant.jl is aiming to do: Take the LLVM code from Julia and pipe to to XLA, where it can benefit from all the investment which make Jax fast.

Same author as the more mature Enzyme.jl, which is AD done at the LLVM level.

Julia's flexibility makes it easy to write a simple AD system, but perhaps we're learning that it's the wrong level to write the program transformations needed for really efficient AD. Perhaps the same is true of "run this on a GPU, or 10" transformations.

> Almost feels like a fallacy of Julia at this point, on the one hand Julia really needs a stable, high-performance AD-engine, but on the other hand it seems to be fairly easy to get a minimal AD-package off the ground.

Nothing wrong with this. Very successful languages like Python and JavaScript has tons of reimplementations of the same things because those languages are _easy_. This isn't a loss for the language it's a victory. If a lot of people can use it that's good.

As a consequence there will be lots of garbage libs, like there are lots of garbage libs in Python and JavaScript.

Reply on news.ycombinator.com