MongoDB Performance for more data than memory | Hacker News Reader

MongoDB Performance for more data than memory

colinhowe.co.uk

34 pointschaostheory20 comments

I don't think it's a MongoDB fault, it is simply true with everything that is serving data bigger than available memory. Of course there is also no solution to the problem as the disk is slower... unless your working set still fits you are going to pay the added latency.

Perhaps I wasn't clear... it's not a fault at all, in fact MongoDB is behaving exactly how you'd expect and hope when the dataset gets too large. I just couldn't find anything out there to confirm this :)

Right, makes sense and experimenting with different systems is the only sane way to make a solid idea about the different solutions... instead of asking on twitter what NoSQL database to use ;)

Btw it's worth noting that there are on-disk databases that will behave pathologically in scenarios where the available memory starts to get seriously smaller than dataset. For instance I saw instances of MySQL that once the data set size reached a given threshold the performance started degradating exponentially. This can probably be avoided.

Exactly, knowing that Mongo isn't one of these (or at least doesn't appear to be so far) is a comfort :)

I assure you MongoDB is just as subject to memory pressure as any other database.

Do you have a sense of how Redis fares in a similar test? I realize there are lots of differences, but I'd be interested in comparison with the same test set, both to see performance drop off as memory fills, and to compare efficiency of memory usage.

In Redis performance is the same regardless of the number of keys, there is no such an issue. But of course in Redis is easy because everything is in memory, and we use O(1) data structures for lookups.

If you use Virtual Memory things are different of course, unless you almost always hit the part of the dataset fitting in RAM. With diskstore there is also to flush things on disk from time to time, so if the site is write-heavy regardless of the amount of memory you'll see a significant slow down.

Basically we think that for write heavy apps the default (in memory) is the way to go. Actually it is a bit different than that: everybody should run Redis in memory, but some specific applications where disk-backed makes sense.

> in Redis is easy because everything is in memory

Only by recommendation. You merely advise users not to allocate a DB larger than available memory. There's no built-in protection against overcommit, at least last time I checked (the documentation has been reorganized recently and I can't find the page that has the command-line arguments anymore).

One could create a capped collection in MongoDB that was smaller than memory to achieve the same effect.

Sure there is, check 'maxmemory' directive. Especially in 2.2 it was extended a lot. You can configure it to simply trow error on writes once the limit is reached.

My point is that Redis doesn't automatically size maxmemory itself. Novice users can trivially make Redis perform awfully if their working set exceeds available RAM.

(I don't consider this a flaw in Redis, BTW - it's just that a lot of implementors really don't understand operating systems and VM overcommit.)

One question I really have is: for the exact same dataset, how much RAM is used to handle it with MongoDB, Redis, MySQL, or even a column-based store.

If someone has such a benchmark already, I'd love to read it.

There's no way to know, really. When you mmap() backing stores into process memory as MongoDB and Redis do, the OS takes care of paging activity for you. And they get to compete with other processes on the system for memory pages.

InnoDB (MySQL) allows you to fix its buffer pool in RAM, so it makes decisions as to which pages to buffer instead of the OS. Sometimes it makes better decisions, sometimes it doesn't.

I had a couple of ideas: given a machine (or a VM) with an amount of RAM which is enough to handle all the db in memory (eg: 8GB for a 512MB dataset), maybe it's possible to track the OS overall used RAM and have a rough idea ?

One other way would be to have a more intimate knowledge of how the stores work and make a computation: for instance when some column stores, you can compute the bytes taken by column if you know how the inner engine works.

You can use the VSZ of the process (from ps(1)) as a rough approximation of memory utilization. This is what the process maximally could address (without mapping or allocating any additional memory) at the moment of measurement, and may include pages that are in fact not in memory at the time (because they might be references to pages that are on files or swap).

RSS tells you what is currently "resident" (i.e. in RAM) but doesn't include pages that are in files or swap.

Personally, I'm much more interested in flows (paging and I/O activity) than stocks (how many pages a process has allocated). If I'm comparing two competing platforms with the same durability guarantees on identical hardware using identical data access patterns, the one with the lower paging/swapping activity wins. It's not always the one with the smaller memory utilization!

While MongoDB does this, I don't think that Redis takes this approach. In theory, it manages the memory itself, reading from disk into its own cache. In practice, I worry that it might end up competing with the system, leading to "full memory" like conditions far before memory is actually exhausted. Thus my interest in how it performs in a simple real life test like this.

No mmap in Redis indeed.

My mistake.

On Linux, at least, that's probably better than mmap()ing pages, since you get better control over when the backing store is synced.

Nevertheless, the risk still exists, as you suggest, that Redis's memory pages could be involuntarily swapped to disk due to memory pressure caused by competing processes.

The big problem with this question is "what's the same dataset"?

The same dataset can be structured multiple different ways. In fact, if your data structure in MongoDB is the same as it is in MySQL, then you're probably structuring it incorrectly.

By the same measure, what are you then testing for? Are you testing for lots of reads, lots of writes? What type of reads? How many servers are you testing across?

MongoDB makes it really easy to denormalize a data structure, so it's easy to "cheat" on the 99% query and perform poorly on the 1% query. This is probably what you want from a real-world performance stand-point, but you have to be really careful about factoring that into your test weightings.

Either way, I'd love to read such a benchmark as well. But to do any justice to the sheer complexity of the task, it probably needs to be a short book.

The problem with Mongo's concurrency approach is that when you introduce writes to the equation (which take out an exclusive lock on the database), the system becomes unusable.

Here's a link to some of the benchmarking work I did. Ignoring the comparison to Clustrix, you can clearly see readers getting starved out when a writer does io.

http://sergeitsar.blogspot.com/2011/01/mongodb-vs-clustrix-c...

Reply on news.ycombinator.com