Pandas also comes off as an unintuitive joke, but my displeasure with it has mostly worn off. Matplotlib however makes me feel angry pretty much everyday.
Pandas also comes off as an unintuitive joke, but my displeasure with it has mostly worn off. Matplotlib however makes me feel angry pretty much everyday.
Tidy verse assumes tidy data. If you are not working with tidy data, it is unlikely to be a big help. Most data can probably be thought of as tidy.
Remember that any and every operation on a data frame returns a data frame, so unlike chaining in Pandas, you never have to worry if a method you want to use belongs to a series or a data frame, or if your method is returning a series or a data frame.
Select() selects columns, filter() selects rows. This never changes unlike the [] which means different thing depending on if it is used on a data frame (which you are not guaranteed to be served after calling a method on data frame in pandas!) on a series or using the .loc or .iloc methods.
There is no index, instead you just filter on rows.
Pandas comes with a ton of build in utilities which the tidyverse doesn’t, mostly because R is already full of functions you can easily apply across columns.
But particularly pandas date handling functions are really cool
My absolute favorite DataFrame library is saddle (for Scala), which I helped write at my old quant job. Very FP oriented and an absolute pleasure to use. Though maybe it’s no surprise that I like something I worked on.
An incomplete list of things that I dislike about Pandas are:
Too many parameters and knobs for each function
Inconsistenty between inplace and copying operations
Unintuitive function names compared to FP
Too much magic in how things work
Functions and parameters accept a wide range of types in order to make things “just work.”
Lots of non-orthogonal convenience functions that do mostly the same thing
I’m not familiar with how “normal” Python is written, but I suspect a lot of the problems come from the abuse of dynamic typing. Dynamic typing allows you to just add more and more levels of crap without actually changing your data/type model. I think there’s a lot of value in “correct” APIs, vs convenient ones.
That being said, Pandas is extremely powerful, and usually very succinct. Maybe not as nice as kdb+/q (nothing really compares for time-series data), but still pretty good.
Usually this is resolved by a complete redesign of the software that gets delayed forever, never becomes production ready and eventually disappears into obscurity.