Hacker News Reader

Doesn't seem like it can handle processing on real-time streams of data. They don't really seem comparable.

From the readme:

"Airflow is not a data streaming solution. Tasks do not move data from one to the other (though tasks can exchange metadata!). Airflow is not in the Spark Streaming or Storm space, it is more comparable to Oozie or Azkaban."

I was replying to a comment not saying airflow is capable of stream processing.

> Not for multinode analytics of huge data sets.

Airflow does this just fine with celery.

That doesn't really seem the same at all.