Hacker News Reader

Comment by kelseyfrog | Hacker News Reader

My data science team evaluated Polars and came back with a mixed bag of results. If there was any performance-critical section, then we would consider employing it, but otherwise it was marginal negative given the overhead of replacing Pandas across dozens of projects.

View on HN

I think that's the right call. IMHO now is the time to experiment with it, not to replace pandas where it's already working.

The API is still seeing some (expected) breaking changes, and could become a maintenance burden across multiple projects. But the API already feels more consistent, and overall seems to be going in the right direction.

You made me curious to look up the recent breaking releases https://github.com/pola-rs/polars/releases/tag/py-0.19.0 https://github.com/pola-rs/polars/releases/tag/py-0.20.0 And their policy about it: https://docs.pola.rs/development/versioning/#deprecation-war...

Looks like you ought to set aside some time to do updates each quarter, but I do wonder how much breakage can there be in practice, most changes seem pretty niche.

Definitely don't rewrite all the code, I think it's worth adopting or evaluating in new code though. Especially with the very cheap pandas interop, you can zero-copy to pandas pretty easily if you use the arrow backend.

Going forward: https://github.com/fugue-project/fugue

Do you know if this project is collaborating with the people behind the "standard data frame API"? https://data-apis.org/dataframe-api/draft/index.html

I do not know but it does not appear so, based on a quick investigation. I had not heard of this API effort myself.

does your team use pandasql or mainly the direct pandas api? curious

Reply on news.ycombinator.com