Hacker News Reader

Comment by svnpenn | Hacker News Reader

> I suggested to a client that they use Sqlite3 for their company database rather than a disk based database.

Sqlite is a disk based database...

Well, maybe not "disk based" per se. It can be stored on disk, yeah. But it can also run in memory. I think in this case it's clear that ChuckMcM meant a machine with 64GB of RAM (not disk space), therefore they are talking about having SQLite running in memory, not on disk.

Even if a DB is disk-based, you can throw it in a ramdisk to get the same speedups, often. I would be curious to know if there are any drawbacks or dangers from doing that, apart from the obvious “if your box powers off, the ramdisk is gone” and “each time the box restarts, you have to copy the db to the ramdisk.”

The main problem I was uneasy about was, I would simply use the equivalent of rsync -Pa /ramdisk /disk in a loop, which seemed to ensure safety in the event of a poweroff. It never broke, but I wouldn’t say it was a good idea, necessarily.

Sure speeds up most things though. The benefits are pretty crazy for I/O bound programs.

Does anyone know the risk of radiation bit flips in this case? I don't think it's an issue with ecc ram, but if it's not that could be a problem for certain applications.

Exactly! I was wondering about that. It seems like it's so rare as not to matter, but that's no guarantee it won't matter.

A friend of mine wrote a 'chaos engineering' tester for this specific issue: http://github.com/enferex/cme

Randomly inject bit flips into a target process memory space, so you can see/test how your software handles it (if at all).

Bit flip is always a risk even if you don't use ram disk.

>I would simply use the equivalent of rsync -Pa /ramdisk /disk in a loop

That's probably a bad idea. rsync isn't going to copy over writes in the same order they were written by the DB. Most DBs assume that when there's some crash that at least the disk would have been in a logically consistent state with respect to the order of certain writes separated by e.g. fsync and barriers.

'The most common way to force an SQLite database to exist purely in memory is to open the database using the special filename ":memory:".'

Reply on news.ycombinator.com