Hacker News Reader

Comment by nirvdrum | Hacker News Reader

How granular are your measurements? I'm certainly not calling your experience into question. But I wanted to relay a cautionary tale of my own.

On one project my team was convinced that Redis was our bottleneck. Profiling was showing that Redis calls were where the most time was spent. I spent some time looking at the redis-rb source and it does a lot of block nesting (4 or 5 levels for every call, IIRC). So, I tried replacing the redis-rb client with something a bit lower level and our throughput doubled. It turned out that all that block processing cost as much as the network call did.

I suppose this isn't all that new: many performance issues are really a "death by 1,000 cuts" situation. Oftentimes your profiler can lead you astray.

View on HN

The performance problems I've had with redis-rb were largely due to my accidentally using it in an improper environment (an event machine based asynchronous service). Replacing it with em-hiredis (and refactoring to use callbacks) drastically improved throughput for us.

In situations where a blocking redis client like redis-rb doesn't interfere with other running code (synchronous web requests), I've found zero performance issues with redis-rb (latency caused by redis is measured in single digits, usually < 5ms).

The tricky part with performance is it's often contextual. I didn't mention it, but I did verify the results by modifying redis-rb to remove the layers of blocks as well. Of course, it could have been something else, like how a particular method handled a block it was provided.

My intention wasn't to rail against redis-rb. It's written very nice, idiomatic Ruby. The faster solution was decidedly less Ruby-like. I was just suggesting that even when your profile looks to be telling you the DB is to blame, you should probably prod a bit more.

ActiveRecord is as another case where I've hit performance issues. I had just swapped out most AR usage for Sequel, so my data on that is probably well out-of-date. But, for us, materializing rows as full DAOs was often a very costly process that could dominate the cost of the DB query.

Yes indeed, context is everything. I've had my own troubles with activerecord as well.

What did you replace redis-rb with, if you don't mind me asking?

In this case we first moved to JRuby (after verifying a similar performance profile to MRI) and once on JRuby we used jedis [1]. I simplified the tale a bit -- apologies if I got your hopes up on MRI.

[1] -- https://github.com/xetorthio/jedis

Reply on news.ycombinator.com