Hacker News Reader

Yes this has been an up-and-coming theme in the data science world. Arrow for the data format, Ibis for the API, Acero/Velox/DataFusion/DuckDB/Polars for execution, Substrait for the query plan representation, etc.

View on HN

Isn't spark already providing a bundled and somewhat functional version of all of this ?

Right, so I think the idea is to "unbundle" this so that you can compose your own data analytics engine.

Reply on news.ycombinator.com