Hacker News Reader

To be honest any file format that would do 1) strong typing/binary data, 2) string deduplication would capture most of the advantages of parquet, except being a column store format which helps in some cases and hurts in others.

View on HN

And even using a columnar format but without all the complexity of Parquet would be comparatively easy (when writing using buffers)

At that point just start using an SQLite database as a data format.

SQLite is a row-oriented store, not a column-oriented store, ie, there are fundamental tradeoffs between the two.

But sqlite has no compression by default.

Reply on news.ycombinator.com