Hacker News Reader

Or you could just warehouse the daily data into something like ClickHouse and start fresh every day. It's built for this kind of workload and has demonstrated some absolutely insane analytical performance at massive scale. We're currently running it on an $170/month VPS, querying over 500+ billion rows daily without any issues. At that point, partitioning an ever-growing OLTP table starts looking like the harder problem.

View on HN

With the other comment about kdb+ and this show a misread of the article: The article AND ANY use of a columnar DB show the same issue: Point lookups are badly pessimized with both partitions on other columns and/or columnar.

That is why columnar DBs, not matter how impressive, are not used by OLTP workloads (and here this article point you can undo the advantages without pay attention at the consequences of partitions).

Well that's not fair, there's been a bunch of tries at HTAP systems (hybridizing both an OLTP and OLAP system with some columnar bolt ons) but there's no free lunch converting between data formats - columnar formats usually have very useful things like run length encoding which is insanely fast to unroll but annoying AF to change.

Sure, the point I try to make is that if you add things useful for OLAP to a OLTP system you could pessimist the OLTP system (that most often is the source of truth)

Reply on news.ycombinator.com