Comment by atrettel | Hacker News Reader

Comment by atrettel | Hacker News Reader

This is an interesting article. I have often wondered if there would be benefits to porting scientific simulation codes to more modern languages. In the end I've always reluctantly stayed in C or Fortran for scientific simulations. The article touched on the some of the issues (and the ever present issue of "If it isn't broke, don't fix it"). But I do think that the article ignored what I think is the biggest issue.

To me the biggest issue is that some simulations just need to run on 1000s or hundreds of 1000s of cores, and the primary technology that lets this happen is MPI (Message Passing Interface). I've been writing new Fortran code only because I want the simulation to be able to scale to this kind of size. The issue is that MPI only has official bindings for C and Fortran, so the only real choice I have is to keep programing in C or Fortran if I want to use MPI.

Any thoughts on this? Personally I would love to move towards more modern languages but without the proper libraries it is still a tough decision.

Rust can bind to C libraries, as can many other languages.

I have looked into this a bit, but the issue appears to be that MPI is not a library per se but a standard interface to more underlying function calls. Different implementations of MPI are in effect different libraries using the same API, and because of that it is difficult to write general bindings rather than just bindings to a particular version of an MPI library that is implemented on a particular supercomputer. In short, it's too site specific and fragile, unfortunately. If you change the system, the program might no longer compile. I appreciate the comment and feedback, though.

This Stack Overflow page comments more on this issue:

https://stackoverflow.com/questions/22949462/rust-on-grid-co...

MPI officially supports only C and Fortran. Anything else for any other language can be only considered as adventures of some open-source developers, that may not last forever. Take a look at the fate of the C++ bindings for MPI.

Including Fortran.

Consider Julia. It has parallel processing built in, and uses MPI under the hood.

A bit more specifically: IIRC it defaults to trying to use MPI when you launch distributed jobs. It can also fall back to using SSH, for "extremely limited cases" and quite frankly it's not inappropriate to use SSH if you are certain that communication is not going to be a bottleneck.

For those sorts of scientific simulations, you probably want to use the MPI.jl package rather than the built-in parallelism – at least that’s the conclusion I came to when I started the fluids code I’m currently working on. MPI.jl is a thin wrapper around the C interface so you end up writing your programs pretty much the same way as in C/Fortran. This type of MPI support is still a bit rough around the edges but it works, and our code runs as fast as the Fortran code it is based on (we usually scale to ~1000 cores). You get the advantage of working with a very nice, modern language, but it is pretty easy to accidentally do things that are very expensive and killing your performance, such as allocating memory inside a hot loop.

We use Akka in java for related distributed computation use cases.

I don't know that I have any thoughts, but I do have

Reply on news.ycombinator.com