Dataflow Programming: Handling Huge Data Loads Without Adding Complexity
drdobbs.com
drdobbs.com
Depending on the case, you might be better off moving the critical part of the data processing to C or C++, perhaps using a relevant library. On many benchmarks Erlang is 10x slower than C. And more importantly for huge data, Erlang consumes 3x to 30x more memory.
http://shootout.alioth.debian.org/u64q/benchmark.php?test=al...
Edit:
Relative to C/C++, Scala is somewhat better in cpu (still slower) but memory-wise it's even worse:
http://shootout.alioth.debian.org/u64q/benchmark.php?test=al...
When I make mistakes reading, it is most often my fault for not paying attention and not reading carefully.
Node.js https://github.com/bergie/noflo
Figure 2 shows run times for "performance of a dataflow based framework" and the writer has already said Erlang uses the actor model not the dataflow model. Not Erlang.
(Probably the Java based dataflow framework sold by the company that employs the writer.)
> The actor model appears much better suited to task-based
> parallelism where concurrency is important. For example,
> Erlang programs control telecom equipment with thousands
> of unique requests per minute. Meanwhile, dataflow is
> focused on data parallelism where single-job data
> processing and analysis performance are the main
> concerns. In a sense, they're slightly different
> solutions for different domains, although conceptually
> similar.
[Citation needed] for all that, with a proper point of reference with C or C++, as per my original comment.