DBMSs do not bypass filesystem. If this was the case, the table names would not be case-insensitive under Windows and case-sensitive under Linux (in MySQL). What they do is to allocate large space on the file system (by the way, the data is still visible as a file / set of files in the underlying operating system) and manage internal data structure. This lower the fragmentation and the overall overhead. In a similar way cache systems works - Varnish allocates entire memory it needs with a single call to operating system, then maintains internal data structure.

>Unless your web app is designed this way (it isn't), you cannot achieve anything close to the throughput of a modern database engine.

Your saying if I write to a section of memory on the heap or stack 4000 times with some different random byte every time it will be slower than doing the same thing to a db? Let's be real.

>Performance is a product of software architecture. Being stateless does not make your software fast in any absolute sense, many stateless in-memory systems have mediocre performance. A disk-backed design with superior architecture can out-perform an in-memory design, both in theory and practice.

Uh fundamental performance of software is absolutely limited by hardware. Stateless removes some fundamental limits. Sure architecture can change things but not not absolutely. Given the best architecture for all cases the best possible performance is in the end determined by which one is stateless. The absolute speed in the end is determined by hardware. SW architecture is changeable and thus NOT absolute.

I would know I work in HPC. We don't have any databases in any of our processing pipelines for obvious reasons.