HNHacker News
TopNewBestAskShowJobs

durner

2 karma · joined May 3, 2024

submissionscomments
durner··on Optimize the Data Layout
Sure, a filtering scan or an index lookup is better in chunked SoA (or PAX as we database people say) than without chunks due to the metadata filter options. We briefly talk about that in the out-of-memory optimization section. Most column-based formats/databases are actually inspired by the ideas of PAX, they often just use a bit coarser granularity (e.g. Parquet's rowgroups).
durner··on Optimize the Data Layout
The idea of comparing column and row storage actually inspired this whole post.