Hacker News Reader

Just for clarification, dask lets you run any python function on any python datatype. Numpy/pandas is faster, but anything is doable with dask. This makes it eminently flexible, unlike many «big data» tools that only work on tables, or other arbitrary limitations.

View on HN

Yes, you can use dask’s lower level APIs, like futures [0] or dask delayed, and it’ll just pickle the python objects. Dask provides a group of collections APIs, like dataframes [1] and arrays that use more efficient serialization methods.

[0]https://docs.dask.org/en/latest/futures.html

[1]https://docs.dask.org/en/latest/dataframe.html

Reply on news.ycombinator.com