Hacker News Reader

Comment by YetAnotherNick | Hacker News Reader

I might be wrong but I don't think any databases operation is limited by sorting. Even naive quicksort with compiler optimization performs very fast and it is much more faster than reading from disk even with extra log n factor. Maybe things like redis sorted sets could benefit from it, but most implementations of things like these are priority queue.

View on HN

Disk read access access is less and less relevant in normal business application DBs.

It is common at the far ends of scalability spectrum: low end where people use $2 VPS, and high end where big tech can afford to invest a lot in engineering effort to build complex systems that are fine tuned for many levels of memory hierarchy.

Most systems can just add memory until data fits. (of course frequently they still don't, and keep doing more expensive things like having many separate copies for different purposes built in various different expensive projects... Sometimes additionally faux justified by using cloud platforms that charge prohitively for memory)

(Hopefully) gone are the days when DBs are spending the bulk of their time blocking in I/O waiting for spinning disks to spin. They spend a lot of intelligence on keeping active tuples paged-in, and memory sizes are exponentially bigger than they used to be, and intelligent query optimization is going to do as much as it can into memory and then apply join algorithms which... are usually sorts.

And even writes, there's sorting there. An LSM or a B-epsilon-tree, etc. they're doing sorting and ordered key traversals before paging out.

It’s pretty common for most SQL queries to be reading data that is entirely cached in memory. In these cases, the performance limitations often come from sorts or hash joins.

Reply on news.ycombinator.com