How to set up a Redis cluster (preview, not production-ready) | Hacker News Reader

How to set up a Redis cluster (preview, not production-ready)

antirez.com

45 pointskristina15 comments

The title is a bit misleading :) This is just a preview of Redis Cluster, it's not good to just claiming that I'm working at it, I wanted to show the first stuff we have that users can actually play with (but not use in production environments!).

Ah, sorry, I assumed people would understand that from the post. I added (preview, not production-ready), is that better?

perfect, thank you.

p.s. I find it very cool that you, working for 10 gen, submitted this article on hacker news. There is still hope in the NoSQL landscape :)

I thought you might find this amusing, but on the EMC network (which I believe VMWare also sits on) your website is blocked -- Uncategorized URL.

Hey antirez, both this and 2.4 look really great. Can you talk to some internals of redis-trib? Is it using a cyclical hash on keys approach? What sort of controls are you planning on giving datastore designers?

I was using redis for consuming a multi-gigabyte financial analysis dataset, but eventually moved on because I didn't like using Hiredis on perl or C, so the clustering is very appealing as a way to keep latency down and get bandwidth higher.

Hey. All the possible keys are simply divided into 4096 slots. You can assign any part of this slots to any given node, so you can assign things in strange ways, like, node 1 50% of all the nodes, and 10% each to node 2,3,4,5,6.

Redis-trib will simply propose something that makes sense, but you will be able to edit the initial guess of the tool.

About sharding before redis cluster you can do it already but with more problems for sure as resharding will be much simpler with redis cluster (however you can just avoid doing resharding for now, using a lot of nodes from the start, not optimal but works). Not sure what is the problem with hiredis or perl, did not understood this part.

hriedis on perl works fine, well, mostly fine. I uncovered at least one small bug which I'll send on at some point.

The issue is that hiredis didn't have a usable python client, and I hated developing in perl, or C. Without hiredis, redis over python is unusably slow for querying 100k+ keys at once, I believe due to python's networking / socket code.

In the end, I ended up moving to pytables for this sort of stuff; it would be nice to use redis for it, but the data access patterns are tuned a little bit more for the web than for a 'do this math on these 3 million data points' kind of use case.

Anyway, I'm excited about redis cluster, I think it fits a huge need for web-scale application development.

Tip: try node/coffeescript and hiredis, you might like it!

Redis cluster will be great when it gets here.

I really like 2 new features in 2.4:

- variadic versions of functions to reduce pipelining. This will reduce latency in my write-happy app

- memory optimized memory sets that will saves me money on RAM.

Thanks Salvatore and Peter!

Reading about Redis Cluster, I am a bit worried that some re-invention of core erlang features is going on.

If nothing else, an erlang to redis bridge might make for a nice pesudo-cluster...

Actually, I don't see any reinvention of Erlang's core features here:

- Erlang: a functional language based on actor model that allows to create distributed applications easily as actors don't share state (plus much more that I won't mention here for the sake of brevity). To share state among various Erlang clusters you have to use something like Mnesia (bundled with Erlang stdlib) or Riak (distributed database built using Erlang).

- Redis Cluster: a way to distribute state among multiple Redis (a simple database application) nodes in a manner transparent for the applications using Redis.

Like execution is more important then idea in a startup, implementation in the right "scope" and with the right tradeoffs is more important than idea in a software system. So while for sure Redis Cluster will look a lot like any other distributed hash table I think it is important that we implement it inside our system, with our system language, and with Redis specific tradeoffs.

I see what you mean, and wish you the best.

Thanks for the reply.

antirez -- looking good. But may I ask why the name 'redis-trib' -- what does the trib mean? Short for contrib? Tribune? All I see is the nsfw definition, but maybe that's just me.

http://www.urbandictionary.com/define.php?term=trib

redis-trib-ute :)

Reply on news.ycombinator.com