Comment by forgotpwd16 | Hacker News Reader

Comment by forgotpwd16 | Hacker News Reader

Some seeings:

- C++ unsurpassable king.

- There's a stark jump of times going from ~200ms to ~900ms. (Rust v1.92.0 being an in-between outlier.)

- C# gets massive boost (990->225ms) when using SIMD.

- But C++ somehow gets slower when using SIMD.

- Zig very fast*!

- Rust got big boost (630ms->230ms) upgrading v1.92.0->1.94.0.

- Nim (that compiles to C then native via GCC) somehow faster than GCC-compiled C.

- Julia keeps proving high-level languages can be fast too**.

- Swift gets faster when using SIMD but loses much accuracy.

- Go fastest language with own compiler (ie not dependent to GCC/LLVM).

- V (also compiles to C) expected it (appearing similar) be close to Nim.

- Odin (LLVM) & Ada (GCC) surprisingly slow. (Was expecting them to be close to Zig/Fortran.)

- Crystal slowest LLVM-based language.

- Pure CPython unsurpassable turtle.

Curious how D's reference compiler (DMD) compares to the LLVM/GCC front-ends, how LFortran to gfortran, and QBE to GCC/LLVM. Also would like to see Scala Native (Scala currently being inside the 900~1000ms bunch).

* Note that uses `@setFloatMode(.Optimized)` which according to docs is equivalent to `--fast-math` but only D/Fortran use this flag (C/C++ do not).

** Uses `@fastmath` AND `@simd`. The comparison supposedly is for performance on idiomatic code and for Julia SIMD is a simple annotation applied to the loop (and Julia may even auto do it) but should still be noted because (as seen in C# example) it can be big.

Author here! Thanks for the detailed breakdown. Let me address a few points:

- C++ SIMD being slower: The standard C++ uses i & 0x1 which lets the compiler auto-vectorize. With -O3 -ffast-math -march=native, gcc/clang do this really well. The explicit AVX2 version has overhead from manual vector setup and horizontal sum at the end. Modern compilers often beat hand-written SIMD for simple loops like this.

- Zig fast-math: Correct. Line 5 has @setFloatMode(.optimized) with a comment saying "like C -ffast-math".

- Julia: Also correct. Uses @fastmath @simd for - both annotations together.

- Crystal/Odin/Ada being slow: All three use x = -x which creates a loop-carried dependency that blocks auto-vectorization. The fast implementations use the branchless i & 0x1 trick instead.

- C# SIMD: Uses Vector512 doing 8 doubles per iteration. That explains the ~4x speedup.

- Nim vs C: Both compile via gcc with similar flags. Probably just measurement variance.

- Fortran: Interestingly does NOT use -ffast-math. Uses manual loop unrolling instead (processes 4 terms per iteration).

- Go: You're right that it's the fastest with its own compiler. No LLVM/GCC backend, just Go's own SSA-based compiler.

For suggestions - DMD, LFortran, and Scala Native would be great additions. PRs welcome!

> curious how D's reference compiler (DMD) compares to the LLVM/GCC front-ends

On my machine (an old i7-8700), dmd performs rather poorly. 3.5 seconds.

    dmd -O leibniz.d
    Measure-Command { ./leibniz.exe } | select -expand TotalMilliseconds
    3535.823

Comparatively ldc runs in 943 milliseconds:

    ldmd2 -O leibniz.d
    Measure-Command { ./leibniz.exe } | select -expand Totalmilliseconds
    943.21

I'm sure there is compiler switch magic that I don't know about that could improve them.

Looking closer at the benchmarks, it seems that C# benchmark is not using AOT, so Go and even Java GraalVM get here an unfair advantage (when looking at the non SIMD versions). There is a non trivial startup time for JIT.

Sorry, I can't seem to edit my answer anymore, but I was mistaken, C# version is using AOT. But the are other significant differences here:

  > var rounds = int.Parse(File.ReadAllText("rounds.txt"));

  > var pi = 1.0D;
  > var x = 1.0D;

  > for (var i = 2; i < rounds + 2; i++) {
  >     x = -x;
  >     pi += x / (2 \* i - 1);
  > }

  > pi \*= 4;
  > Console.WriteLine(pi);

For example, if we change the type of 'rounds' variable here from int to double (like it is also in Go version), the code runs significantly faster on my machine.

Try that on ARM64 and the result will be the opposite :)

On M4 Max, Go takes 0.982s to run while C# (non-SIMD) and F# are ~0.51s. Changing it to be closer to Go makes the performance worse in a similar manner.

Reading the repo, the benchmark includes the entire program execution from startup to reading the file.

For the sub-second compiled languages, it's basically a benchmark of startup times, not performance in the hot loop.

How much difference would it make for these tiny programs?

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Author here. I just tested this with Zig by timing the segments inside the program:

  1 billion iterations:
  - Startup + file read: 0.01ms
  - Computation: ~200ms
  - Overhead: 0.01%

Even at 1 million iterations (0.24ms total), startup is only 4% overhead. At 1 billion it's essentially zero.

The benchmark is definitely measuring the hot loop, not startup time.

> Go fastest language with own compiler (ie not dependent to GCC/LLVM).

C# is using CoreCLR/NativeAOT. Which does not use GCC or LLVM also. Its compiler is more capable than that of Go.

Reply on news.ycombinator.com