Consistent hash ring Python library for building distributed apps with Redis

github.com

31 points·thomas-st··6 comments

The hash ring is so dumb. I don't understand why it's still a thing (or was a thing to begin with.)

Just hash and mask some number of bits, and map each index you get to some node. Faster, simpler, even enough distribution.

If your number of nodes is not a power of two, map multiple buckets to each node.

When adding nodes, and the # buckets is too low, split each bucket in two by adding one bit to the mask.

Consistent table construction means you only need the set of nodes to construct the take, keeping lookups entirely in core.

Maybe I should blog this, but it feels a bit like "Hey you cab implement modulo power of two with a bit mask! Isn't that so crazy?"

You have invented kademila!

As long as the network is "fully connected" by knowing all other members, topology will not really matter.

If you use a Chord or pastry style skip list, you again have the exact same metrics as kademlia.

All of these redis structures are really just re-branded DHTs.

What's "dumb" about the hash ring, and how exactly is your solution faster/simpler?

One nice thing about the hash ring is that it nicely rebalances when you add or remove nodes. There is no need split any buckets.

why not use Jump consistent hash?

This is good - but isn't this problem is now solved with redis cluster?

Redis cluster is clustering for Redis; this is clustering for a certain class of other things, built on top of Redis.