Hacker News Reader

It is optimized for analytic workloads (default in-mem columnar layout), whereas PG and sqlite are OLTP DBs. It's going to be insanely faster for those use cases.

And with DuckDB?

DuckDB is an in process OLAP, I’ve been using it a lot, and I am keen to use Polars but DuckDB seemings to be flying for me.

I like Polars’ syntax, a lot of times its dataframes API is much nicer than SQL.

It’s much more similar to cleaned up Spark syntax vs Pandas’ strange API.

I’m quite happy with things taking 3 minutes instead of an hour or so.

Actual databases pay up to two orders of magnitude of speed for durability. If you can regenerate your dataset in case the DB drops it (or there is a power outage, or whatever), being able to complete a complex, write-heavy query in 10s instead of 15 minutes is actually very useful when doing analysis.

Because those are databases? Polars is a data processing engine, not a database. They have different uses.

If you want to use a database instead of Polars, for similar use cases, duckdb is a much better bet than postgressql or sqlite.