Hacker News Reader

Comment by ajtaylor | Hacker News Reader

I was thinking you had switched from string keys to hashes, since I had read a post from antirez about hashes being more efficient for this type of thing. But as they say, measure twice, cut once. After all, profiling is the first step of optimisation.

View on HN

Hashes can be efficient because redis compresses hashes that match certain restrictions (eg: under 1000 items, each item being under 1000 characters long) and the same can apply to lists, sets, and zsets.

This compression may have inadvertently been activated by the author when he pruned his zsets, which is why the savings are so apparent.

The primary reason for the reduction was going from ~7k items in the ZSET to 100 for several thousand ZSETS. That's not to say the built-in compression didn't affect anything, but obviously less data == less memory.

This was one of the frustrating parts of trying to fix my issue. Every search I did for Redis optimization tips said to use hashes to reduce redundant keys (and, in doing so, use a memory efficient data structure). But every key my library uses is a SET or ZSET so, of course, hashes were off the table for me.

The optimization trick is outlined here:

http://redis.io/topics/memory-optimization

If your sets and zsets are bounded within a certain limit then they will be compressed and take up quite a bit less space.

Updating your configuration obviously requires a restart so that will need to be kept in mind (ie: if your data fragmentation is high then restarting will eliminate the fragmentation, so space savings will be apparent but they're not actually indicative of any changes).

Reply on news.ycombinator.com