Hacker News Reader

Someone says "hey if we had 900GB of RAM we could make a lot of money" and then someone else says "that's ridiculous and impossib- hang on a minute" and scurries off to hack together some tech heresy over their lunch break.

By the way, you can find single servers with 32TB of ram nowadays.

Honestly it’s not too far off from what many databases do if they can. They manage one giant file as if it’s their own personal drive of memory and ignore the concept of a filesystem completely.

Obviously that breaks down when you need to span multiple disks, but conceptually it really is quite simple. A lot of the other stuff file systems do are to help keep things consistent. But if there’s only one “file“ and you don’t ever need metadata then you don’t really need that.

Very smart solution really.

Yeah, a lot of database storage engines use O_DIRECT because the OS's general purpose cache heuristics are inferior vs them doing their own buffer pool management. That said if you try this naively you're likely to end up doing something a lot worse than the Linux kernel.

If I had to guess:

Doing it this way = $

Doing it that way = $$$

it's a very reasonable thing to do if you need performance.