Hacker News Reader

Do you have any data on bringing new nodes online / offline? Obviously, rebalancing in general is going to be I/O-intensive if you have a ton of data, but has Tempest shown any upside/downside when it comes to scaling a cluster?

Like, Tempest helped you with your static cluster... but could it make adding a new node slower or faster?

View on HN

We've tested for stability when adding and removing nodes, but haven't compared the time-to-balance of tempest versus the default balancer. Because an ES cluster remains fully functional (you still have access to all data) while a rebalance is in progress, we chose to optimize for resource usage rather than time-to-balance. There's not really even a good way to compare the balancers' time-to-balance, since they're both highly configurable (range_ratio and iterations in tempest's case, the 4 balance weights in ES's case) - default values probably aren't "equivalent" in terms of time-to-balance, since it's a very minor concern compared to resource usage and stability.

Reply on news.ycombinator.com