Hacker News Reader

Comment by polskibus | Hacker News Reader

How does this differ from Threading Building Blocks ? https://www.threadingbuildingblocks.org/

One can still use TBB inside the functions to make use of it alongside getting MPI parallelism from easyLambda.

Moreover, easyLambda has data flows, map, reduce which in a way improves the way programs are written and composed. There are no classes to be declared for anything. It has many syntactic sugars, for column selection, control over parallelism, i/o etc. I personally like the style of coding in it for various problems, keeping parallelism aside.

Since it supports distributed computation does it have any abstraction for dealing with failures, for example, network problems etc. Support for checkpointing or some other abstraction that deals with these kinds of errors would be very useful. In fact absolutely essential for anything of non-trivial scale.

BTW not complaining, far, far from it, just a useful feature addition you may consider in case you have the bandwidth to invest.

Yeah, you are right, fault tolerance is a needed feature. Although, I'm not pretty sure if that needs to be added to easyLambda itself. Since, MPI 3.0 is expected to support fault tolerance by default. But instead of waiting for MPI 3.0 fault tolerance support, I'm thinking of running it on top of HPX or some other system that provides good runtime support on top of MPI. Also, since most of parallelism scheme is confined to MPIBridge which is just a unit in the flow, these different implementations can coexist.

By skimming through the code, this seems to have the same data-flow model as Intel's TBB, but extended to distributed memory configurations using MPI as backend.

One big difference is that easyLamdba has MPI parallelism, i.e. you can run it on lots of machines, whereas TBB is for a single machine.

Reply on news.ycombinator.com