HNHacker News
TopNewBestAskShowJobs

timhaines

177 karma · joined May 14, 2012

Cofounder of Percy.io http://twitter.com/timhaines
submissionscomments
timhaines··on Migrating From MongoDB To Riak At Bump
Feel free to investigate further. I had to move on.
timhaines··on Migrating From MongoDB To Riak At Bump
Hey - the Basho guys were aware and reproduced it pretty quickly. They saw the same response from their new bloom filter branch they're introducing soon too.

I was monitoring with iostat and a couple of other tools. It was certainly very heavy on io, with 80% util, 20% iowait, and that increased as the currency went up.

I was using protobuf, and a w value of 1, so I was out of things to optimize.

When I was inserting objects already in Riak's cache, it ran about 3 times faster, but of course that's not possible with new objects.

timhaines··on Migrating From MongoDB To Riak At Bump
The benchmarking tool is very slick. Easy to configure for a variety of scenarios, and once you figure out how to install R it produces those pretty graphs.
timhaines··on Migrating From MongoDB To Riak At Bump
Well, for the benchmark, I was using Basho's benchmarking tool which is erlang, and I was testing with protobuf. I had 5 concurrent clients running for the benchmark, but also tried with more and less, and got about the same results.

Search wasn't in use on the test bucket.

For my app, I'd integrated Riak using ruby.

timhaines··on Migrating From MongoDB To Riak At Bump
If you're thinking about using Riak, make sure you benchmark the write (put) throughput for a sustained period before you start coding. I got burnt with this.

I was using the LevelDB backend with Riak 1.1.2, as my keys are too big to fit in RAM.

I ran tests on a 5 node dedicated server cluster (fast CPU, 8GB ram, 15k RPM spinning drives), and after 10 hours Riak was only able to write 250 new objects per second.

Here's a graph showing the drop from 400/s to 300/s: http://twitpic.com/9jtjmu/full

The tests were done using Basho's own benchmarking tool, with the partitioned sequential integer key generator, and 250 byte values. I tried adjusting the ring_size (1024 and 128), and tried adjusting the LevelDB cache_size etc and it didn't help.

Be aware of the poor write throughput if you are going to use it.

← PreviousPage 2 of 2