markkitti

Via Pluto.jl notebooks, I have been able explore the Julia-JavaScript interface. It's quite fun using JavaScript and SVG to interactively render the accelerated BLAS and FFTW outputs from Julia.

I thought that notebook based development and package based development were diametrically opposed in the past, but Pluto.jl notebooks have changed my mind about this.

A Pluto.jl notebook is a human readable Julia source file. The Pluto.jl package is itself developed via Pluto.jl notebooks.

https://github.com/fonsp/Pluto.jl

Also, the VSCode Julia plugin tooling has really expanded in functionality and usability for me in the past year. The integrated debugging took some work to setup, but is fast enough to drop into a local frame.

https://code.visualstudio.com/docs/languages/julia

Julia is the first language I have achieved full life cycle integration between exploratory code to sharable package. It even runs quite well on my Android. 2023 is the first year I was able to solve a differential equation or render a 3D surface from a calculated mesh with the hardware in my pocket.

Julia 1.9's native precompilation is definitely helpful in that regard but loading those native shared libraries (.so files on Linux) into Julia does take some to verify.

If the main objective is to reduce time to load and time to first task then PackageCompiler.jl [3] is still the ultimate way to do so.

Because Julia is a dynamic language, there are some complicated compilation issues such as invalidation and recompilation that arise. Adding new methods or packages may result in already compiled code no longer statically dispatching correctly requiring invalidation and recompilation of that code.

It slightly more complicated than what you stated. It's "I want to compile and cache native code for the following methods on the following types in this particular environment". PackageCompiler.jl can wrap the entire environment into a single native library, the system image.

[3] https://github.com/JuliaLang/PackageCompiler.jl

That total timing also includes compilation, not just the JSON parsing.

C is not the fastest language. Fortran can often beat them because is has non-aliasing function arguments and does not treat arrays like pointers.

https://fortran-lang.discourse.group/t/fortran-is-faster-tha...

I recall being in a Babbages and examining a Windows 95 box on the shelf. What a different time.

The first annual US Research Software Engineering conference will be held in October 2033 at the University of Illinois at Chicago to discuss software enabled discovery and beyond.

There was a release 25 days ago. The HN submitter might even be the author.

The comparison in the README is to the parquet package for R.

Over time, I've found it is quite possible to mitigate most of these issues. The default settings are not great, mostly because they default to maximize backwards compatibility. With a few settings you can consolidate the metadata such that the data layout is very simple.

The structure of a HDF5 file can be manipulated to put data where you want it in a file. For many files of the same type, you can set the metablock size so that your datasets are always at the same offsets.

You can use tools like h5ls or h5dump to describe the structure of a HDF5 file. It turns out that if you do this, you can make HDF5 cloud friendly.

https://medium.com/pangeo/cloud-performant-netcdf4-hdf5-with...

Surely Python would not skip the 2.

There seems to be significant activity around ASML in the Netherlands: https://info.juliahub.com/sciml-asml-lounge-eindhoven-meetup

See https://docs.julialang.org/en/v1/manual/workflow-tips/#Revis...

A lot has changed in a few years. This release is a big one.

1. I run Julia on my smartphone and often use it as calculator.

2. You typically only need Plots.jl for most needs. See https://docs.juliaplots.org/stable/

3. See https://juliaacademy.com

Another alternative environment are Pluto notebooks. It's reactive like a spreadsheet, but easy to use in your browser.

https://featured.plutojl.org/

I have several users without much coding experience using Pluto notebooks just to generate plots from CSV files. They are finding the combination of a web based interface, reactive UI, and fast execution easier to use than a MATLAB Live script.

You may be confusing front end APIs and the compiler backends.

Julia is flexible enough that you can essentially define domain specific languages within Julia for certain applications. In this case, we are using Julia as an abstract front end and then deferring the concrete interface to vendor specific GPU compilation drivers. Part of what permits this is that Julia is a LLVM front end and many of the vendor drivers include LLVM-based backends. With some transformation of the Julia abstract syntax tree and the LLVM IR we can connect the two.

That said we are mostly dependent on vendors providing the backend compiler technology. When they do, we can bridge Julia to use that interface. We can wrap Vulkan and technologies like oneAPI.

https://github.com/JuliaGPU/Vulkan.jl https://github.com/JuliaGPU/oneAPI.jl

As for syntax, Julia syntax scales from a scripting language to a fully typed language. You can write valid and performant code without specifying any types, but you can also specialize methods for specific types. The type notation uses `::`. The types also have parameters in the curly brackets. The other aspect that makes this specific example complicated is the use of Lisp-like macros which starts with `@`. These allow for code transformation as I described earlier. The last aspect is that the author is making extensive use of Unicode. This is purely optional as you can write Julia with just ASCII. Some authors like to use `ε` instead of `in`.

This is slightly premature. They only just tagged the release on Github several hours ago. While it does suggest the actual release is imminent, it's not really official until you see here on Discourse:

https://discourse.julialang.org/c/announce/25

I thought this might be related to the prior Julia to wasm fluid dynamics simulation, but it seems to be independent of that effort.

https://alexander-barth.github.io/FluidSimDemo-WebAssembly/

Julia is finally going to start storing generated native code on disk in Julia 1.9:

https://github.com/JuliaLang/julia/pull/47184

Here's the concept. Perhaps we have the order wrong. Maybe the physical factor should be the primary factor. The second factor should never be transmitted, but rather is used to unlock the physical factor.

I'm confused by this statement because AOT compilation is an option, and it's getting better over time.

https://julialang.github.io/PackageCompiler.jl/stable/sysima...

In Julia 1.9 and beyond, this process now also becomes modular with a native library being produced per package.

The difficulty is determining what exactly must be compiled since Julia's methods are polymorphic. There's are tools that exist to help with this. One of the latest that is SnoopPrecompile.jl.

The presence in a journal is not a grade. It is a "pass".

Perhaps some journals only let grade "A" papers pass, but sometimes that same journal has an editor who might let their buddy's article slip through. If one is part of the old boys club, this is a nice situation.

What eLife wants are signed grades from editors and reviewers. The editor is supposed to attach a brief summary in a few sentences summarizing the reviews. Hopefully one might bother to read a sentence or two to evaluate a paper instead of merely looking at the cover of the journal.

eLife's move here is basically a statement that the system is corrupt. Those wanting to fight the corruption are trying to increase transparency and reduce arbitrary decisions.

OME is the best organization we have for standards in microscopy, especially bioimaging.

There may be some confusion about which MIT license we are talking about as there is more than one that originates from MIT or has MIT in the name.

When people refer to "MIT License" they usually mean the Expat License:

> Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

> The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

https://en.wikipedia.org/wiki/MIT_License?wprov=sfla1

I am not a lawyer. Anyone interested in what they can or cannot do with AGPL licensed software should probably consult a lawyer.

It may depend on what you mean by use an API. Compiling a program that links AGPL licensed code may incur obligations under the AGPL. Here is a quote from the AGPL FAQ.

> If the modules are included in the same executable file, they are definitely combined in one program. If modules are designed to run linked together in a shared address space, that almost surely means combining them into one program. https://www.gnu.org/licenses/gpl-faq.html#MereAggregation

Now with better formatting...

This becomes an important part of optimizing Julia code. There is some tooling for this. Below, identity is a type stable function because we know that an Int64 argument results in a Int64 output. The macro code_warntype reveals the type analysis:

  julia> @code_warntype identity(5)
  MethodInstance for identity(::Int64)
    from identity(x) in Base at operators.jl:526
  Arguments
    #self#::Core.Const(identity)
    x::Int64
  Body::Int64
  1 ─     return x


  julia> f(x) = identity(x ≥ 0 ? x : x + 0.0) 
  f (generic function with 1 method)

  julia> @code_warntype f(5)
  MethodInstance for f(::Int64)
    from f(x) in Main at REPL[6]:1
  Arguments
    #self#::Core.Const(f)
    x::Int64
  Locals
    @_3::Union{Float64, Int64}
  Body::Union{Float64, Int64}
  1 ─ %1 = (x ≥ 0)::Bool
  └──      goto #3 if not %1
  2 ─      (@_3 = x)
  └──      goto #4
  3 ─      (@_3 = x + 0.0)
  4 ┄ %6 = @_3::Union{Float64, Int64}
  │   %7 = Main.identity(%6)::Union{Float64, Int64}
  └──      return %7

In Julia I do not need the Godbolt compiler explorer. The macros `@code_llvm` and `@code_native` show me the LLVM IR and native code for a function.

  julia> @code_llvm debuginfo=:none 5.0 + 3
  define double @"julia_+_156"(double %0, i64 signext %1) #0 
  {
  top:
    %2 = sitofp i64 %1 to double
    %3 = fadd double %2, %0
    ret double %3
  }

  julia> @code_native debuginfo=:none 5.0 + 3
  ...
    vcvtsi2sd %rdi, %xmm1, %xmm1
    vaddsd %xmm0, %xmm1, %xmm0
    retq
  ...

We already have some forward prototypes of being able to run Julia ahead-of-time compiled native code from the command line.

https://github.com/brenhinkeller/StaticTools.jl

I think what we'll end up with is a language that can be used in both a fully static mode and in a dynamic mode along with some possible mixing. We may yet get the benefits of a statically compiled language as the tooling continues to develop. I do not see anything inherent in the language that would prevent that from happening.

Julia 1.9, now in beta, implements native code caching. Precompiling a Julia package now creates a native shared library, a ".so", ".dylib", or ".dll" file. For some packages, this lowers load time considerably. It may some time before many packages take full advantage of this.

The promise of Julia is that you can have the high-level interface and the low-level code in the same language. The alternative would be coding the low level code in Rust or C and then creating bindings for Python or R.

For a while Julia made the most sense for long-running code that is that is executed almost as often as it is modified (e.g. scientific computing). In this situation Rust or C static compilation times become a hinderance. As ahead-of-time and static compilation features get added to Juliaz this scope will expand.

This becomes an important part of optimizing Julia code. There is some tooling for this. Below, identity is a type stable function because we know that an Int64 argument results in a Int64 output. The macro code_warntype reveals the type analysis:

``` julia> @code_warntype identity(5) MethodInstance for identity(::Int64) from identity(x) @ Base operators.jl:513 Arguments #self#::Core.Const(identity) x::Int64 Body::Int64 1 ─ nothing └── return x ```

This is type unstable and results in dynamic dispatch because we are not sure if the argument to identity will be an Int64 or a Float64.

``` julia> f(x) = identity(x ≥ 0 ? x : x + 0.0) f (generic function with 1 method)

julia> @code_warntype f(5) MethodInstance for f(::Int64) from f(x) @ Main REPL[4]:1 Arguments #self#::Core.Const(f) x::Int64 Locals @_3::Union{Float64, Int64} Body::Union{Float64, Int64} 1 ─ %1 = (x ≥ 0)::Bool └── goto #3 if not %1 2 ─ (@_3 = x) └── goto #4 3 ─ (@_3 = x + 0.0) 4 ┄ %6 = @_3::Union{Float64, Int64} │ %7 = Main.identity(%6)::Union{Float64, Int64} └── return %7 ```

Multiple dispatch is not unique to Julia, but it is a large part of the language. It helps 3rd parties extend rather than duplicate interfaces.

There a few recordings of the "Unreasonable effectiveness of multiple dispatch" talk that explains this: https://youtu.be/QTCKsqIK6nE