I'm interested in this general approach being implemented in so many newer engines (those which Onyx apparently competes with) - the distributed pipeline processing/directed acyclic graph (DAG) approach. So many of these engines have related components - message queues, central coordinator, etc.

It seems this is the current popular technique (and rising in popularity) to solving some really long-standing, hard, data processing problems and related performance concerns - really an extension beyond the progress we made with distributed MapReduce algorithms and the like in Hadoop.

There are certainly a lot of implementations coming out - why would one choose Onyx over say, something like Spark? Is it just the compatibility with the Clojure programming language and the associated, implied benefits of functional programming paradigms (among others?).