Hacker News Reader

Comment by adastra22 | Hacker News Reader

Numpy is Fortran btw.

Numpy is most importantly BLAS and LAPACK, which can be implemented in any language. The best CPU version is probably still MKL, which is a mix of Fortran, C, and assembly. C is used for these kinds of projects because it is easy to sprinkle assembly into C.

Fortran solves a particular problem: produce a pretty good binary from readable code that a normal PHD student or engineer can write. This is not really the problem that a tuned BLAS/LAPACK library has to face, vendors throw very experienced folks at these libraries to chase after those last 5% performance boosts that Fortran leaves on the table.

From numpy.org:

> The core of NumPy is well-optimized C code.

Amusingly, "well-optimized C code" includes calling out to fortran libraries. :D

https://numpy.org/doc/1.13/user/building.html

It's turtles all the way down, except for the final turtle, which is Fortran...

The last turtle is probably assembly, or at least intrinsics, in most good BLAS implementations. Probably called from C, because that’s easier in C.

The nice thing about Fortran is that you get something almost as good, but a normal engineer can write it. There are more C programmers than Fortran programmers, but there are more Fortran programmers than there are C programmers who can write really good assembly kernels. And all the C programmers in that last group are already working on vendor BLAS implementations.

That's a 6 year old documentation page, modern numpy does not use fortran in any way except that maybe you have a particular BLAS reference implementation.

Numpy is basically a wrapper to BLAS and LAPACK though, so it seems a bit odd to ignore the fact that those system libraries are usually written in Fortran.

The reference implementation is written in Fortran, but I'm not sure about the implementations actually used. OpenBLAS has fortran components, but I believe cuBLAS and MKL are vast majority C/C++.

This is a very fair challenge. My understanding was that the main ones in use were fortran and it is very interesting to see that may not be the case. Love to see any benchmark or other comparisons on the options, if you know of any good links.

cuBLAS is not Fortran because it targets NVIDIA GPUs. Despite what the grandparent says, MKL does include some Fortran in its codebase. Most other vendor-provided BLAS and LAPACK libraries are based on the reference Fortran implementation. Even OpenBLAS, the open-source reimplementation, has large parts written in Fortran.

I did link to the old one.... but if you follow the link to the new, it still has that section? I'm confused.

The old one says you need a fortran compiler to build numpy, the other one explicitly says you don't need a fortran compiler to build numpy.

The new one says it auto selects. I don't think they got rid of using the fortran if it is there. They just automated a lot of that in the build.

The well-optimized part is C calling Fortran. :)

no?

If you are using any of the faster libraries for it, it is? Right?

I don't believe that numpy requires any fortran [0] and most BLAS implementations are not written in Fortran either.

[0]: https://github.com/search?q=repo%3Anumpy%2Fnumpy+path%3A**%2...

Most BLAS and LAPACK implementations are written in Fortran. There are some C/C++ alternatives, but those are the exceptions (even if some are widely used in the FOSS community) and they still don’t benchmark as well even in 2023.

cuBLAS benchmarks pretty well :) but sure, I am not super familiar with the state of CPU BLAS frameworks so I will take your word for it.

The good BLAS libraries are MKL and BLIS nowadays, I’m pretty sure.

BLIS is C and assembly if I remember correctly.

MKL is a big library, Wikipedia lists the languages used as C, C++, Fortran, and I think it is also pretty well known that they’ve sprinkled some magic assembly in there.

I believe MKL is mostly C/C++, although true extent is not really knowable.

The reply about cuBLAS was tongue in cheek as it is a GPU BLAS library

Intel hired Mr. Goto a while ago, he wrote gotoBLAS (from which openBlas is derived, so this one dude is responsible for a ton of FLOPs).

The thing that made gotoBLAS good was the hand-tuned assembly kernels, so we can be reasonably sure that MKL has hand tuned assembly kernels at this point.

I think the tongue-in-cheek comment is actually really good. It is a reminder that BLAS is more like a linear algebra API than a particular library.

Only someone at Intel could tell us for sure, but my understanding is that MKL is a C/C++ interface to a rather bog-standard BLAS/LAPACK library written in Fortran, but with various inner loops replaced or rewritten at runtime with a selection of assembly-optimized x86 SIMD codes chosen based on the capabilities of the CPU.

I’m sure it depends on the subroutine.

Reference BLAS is of course Fortran but nobody would actually use it. Some other good BLAS are (in order of historical to cutting edge), ATLAS, OpenBLAS, and then BLIS. They all mostly use C and assembly.

BLIS makes a good BLAS. It almost keeps up with MKL, according to the nice plots on their GitHub.

https://github.com/flame/blis/blob/master/docs/Performance.m...

They use C mostly, because the magic is in the GEMM (and possibly TRSM) kernels, which you are going to write in assembly anyway.

Intel also released oneDNN open source, which has an awful lot of BLAS-like functionality. No Fortran in there.

MKL is older than all these libraries of course (maybe not ATLAS, I’m not sure actually), and has a lot of other functionality (sparse stuff, FFTs), so I’m sure it has more Fortran! But it would be atypical compared to the rest of the BLAS implementations, if, like, their DGEMM was Fortran.

I think I group hand-tuned assembly kernel and C in my mind because of the syntax sugar that C provides.

That’s fair practically, but in the eternal battle between conventional HPC languages and C, we can’t give C an unmitigated win. :)

https://numpy.org/doc/stable/user/building.html#choosing-the... is the latest stable, and still has fortran?

> While a FORTRAN 77 compiler is not necessary for building NumPy, it is needed to run the numpy.f2py tests. These tests are skipped if the compiler is not auto-detected.

Reply on news.ycombinator.com