How about Dask - which is fairly production grade and has experimental Arrow integration.
https://github.com/apache/arrow/blob/master/integration/dask...
Dask deploys pretty well on k8s - https://kubernetes.dask.org/en/latest/
https://github.com/apache/arrow/blob/master/integration/dask...
Dask deploys pretty well on k8s - https://kubernetes.dask.org/en/latest/
Semi-related question: How do you expect Arrow to be integrated to the larger data science landscape?
Will it mostly be used as a go between format? Will new libraries using it internally and old libraries just reading and translating it to a native format? Do you think established libraries will change their back-end to arrow? Is that even feasible with e.g. Pandas (or are you too far from their governance now to say)?
Thanks for the link. I will read about their Kubernetes support.