Btw it's worth noting that there are on-disk databases that will behave pathologically in scenarios where the available memory starts to get seriously smaller than dataset. For instance I saw instances of MySQL that once the data set size reached a given threshold the performance started degradating exponentially. This can probably be avoided.
Here's a link to some of the benchmarking work I did. Ignoring the comparison to Clustrix, you can clearly see readers getting starved out when a writer does io.
http://sergeitsar.blogspot.com/2011/01/mongodb-vs-clustrix-c...
If someone has such a benchmark already, I'd love to read it.
InnoDB (MySQL) allows you to fix its buffer pool in RAM, so it makes decisions as to which pages to buffer instead of the OS. Sometimes it makes better decisions, sometimes it doesn't.
One other way would be to have a more intimate knowledge of how the stores work and make a computation: for instance when some column stores, you can compute the bytes taken by column if you know how the inner engine works.
RSS tells you what is currently "resident" (i.e. in RAM) but doesn't include pages that are in files or swap.
Personally, I'm much more interested in flows (paging and I/O activity) than stocks (how many pages a process has allocated). If I'm comparing two competing platforms with the same durability guarantees on identical hardware using identical data access patterns, the one with the lower paging/swapping activity wins. It's not always the one with the smaller memory utilization!
Nevertheless, the risk still exists, as you suggest, that Redis's memory pages could be involuntarily swapped to disk due to memory pressure caused by competing processes.
The same dataset can be structured multiple different ways. In fact, if your data structure in MongoDB is the same as it is in MySQL, then you're probably structuring it incorrectly.
By the same measure, what are you then testing for? Are you testing for lots of reads, lots of writes? What type of reads? How many servers are you testing across?
MongoDB makes it really easy to denormalize a data structure, so it's easy to "cheat" on the 99% query and perform poorly on the 1% query. This is probably what you want from a real-world performance stand-point, but you have to be really careful about factoring that into your test weightings.
Either way, I'd love to read such a benchmark as well. But to do any justice to the sheer complexity of the task, it probably needs to be a short book.
If you use Virtual Memory things are different of course, unless you almost always hit the part of the dataset fitting in RAM. With diskstore there is also to flush things on disk from time to time, so if the site is write-heavy regardless of the amount of memory you'll see a significant slow down.
Basically we think that for write heavy apps the default (in memory) is the way to go. Actually it is a bit different than that: everybody should run Redis in memory, but some specific applications where disk-backed makes sense.
Only by recommendation. You merely advise users not to allocate a DB larger than available memory. There's no built-in protection against overcommit, at least last time I checked (the documentation has been reorganized recently and I can't find the page that has the command-line arguments anymore).
One could create a capped collection in MongoDB that was smaller than memory to achieve the same effect.
(I don't consider this a flaw in Redis, BTW - it's just that a lot of implementors really don't understand operating systems and VM overcommit.)