Hacker News Reader

Comment by chaps | Hacker News Reader

100% my experience whenever I work with it, and I've been working with it on and off for about five years. I get its appeal -- it fits that fuzzy place where a database is too heavy, but vanilla python is cumbersome... but damn, is it tough to work with. With almost no fail, I always seem to just scrap the pandas code in favor of vanilla python, either from usability issues, or from yet another a library's false promise that it works well with dataframes. So many hours lost in using pandas.

That said, one of its best features, and probably the only thing I use it for these days is, `pd.read_sql(sqlstr, conn).to_csv(fp)`. This is far less cumbersome than using psycopg2.

Edit: for charting these days, similar to OP's visualiztions, I highly recommend vega-lite.

View on HN

Funny that the example you give is one of the (many) frustrations with Pandas.

It’s ‘read_<format>’ to read something in, but ‘to_<format>’ to write it out? In which world is this intuitive? Surely it should be read/write or even from/to?

It's `pandas.read_<format> -> DataFrame` and `DataFrame.to_<format>`. So pandas reads a data format and gives you a dataframe. The `to_<format>` are instane methods of the DataFrame where you write a dataframe to a format. To me that makes sense.

Sure, you can create a sort-of logic to fit the existing naming, but it's still something you have to learn.

Imagine you were using Pandas for the first time, working through a tutorial, and you've just learned that pd.read_csv reads a csv into a dataframe.

Intuitively, what would you expect the corresponding output function to be called? I'd hazard that a vast majority of people would guess at some version of write_csv, and would experiment with either pd.write_csv, or df.write_csv.

Pandas isn't great, but I find it significantly nicer to use than normal Python. Why do you want to write a 100 lines of dumb loopy code when you can probably solve the same problem with a couple lines of Pandas?

If you really just want to use loops and stuff (which I would discourage) just use a list or a dict or something.

As you indicate, Pandas is effectively another language, distinct from regular Python. E.g. it doesn't really use loops. So I think what irritates a lot of people about pandas is that they think "Cool. I can solve this complex problem in a couple of lines of Python code with pandas." then get irritated when they find that the couple of lines of pandas they need to use is "Incomprehensible pandas gibberish" instead of the familiar Python code they were expecting.

"If you really just want to use loops and stuff (which I would discourage) just use a list or a dict or something."

Like my post said, I favor vanilla python over pandas... so yes, I use lists, dicts and somethings. FWIW, though, my workflow pushes everything into postgres, and things that would normally go into pandas are just accessed through SQL through and with helper functions.

Reply on news.ycombinator.com