Scaling Your Clouds

justcramer.com

31 points·zeeg··4 comments

> There are many amazing systems out there that are built on the notion of distributed data with the goal of some level of horizontal scalability (Riak, Cassandra). They also do not allow youto spin up more servers and gain more capacity immediately.

I thought that was exactly what cassandra does? I haven't used it myself, but I am investigating it, and I'd love to hear more on this point...

Cassandra, Riak, et all, are basic distributed systems with the notion of easing horizontal scalability. That said, they do not provide instant (or even close to instant) turnaround on bringing new nodes online. Even more so, you're still limited in how you structure your data.

It may sound like I'm criticizing them, but they're still amazing, and IMO where the future of distributed datastores are going.

Right, for example when you introduce a new server into a MongoDB replica set, it won't instantly become active if you have a lot of data, it needs to sync everything before it becomes active. The key with these systems is too scale early, before the load gets out of hand. Also, MongoDB replica sets can only have a maximum of 12 members.

A sudden spike in traffic which exceeds your planned capacity means that not only are your servers running at 100% but now they need to also dedicate a portion of their now-fully-committed capacity to syncing their data to the new server(s). You might be able to bootstrap this process partially by priming a new server from backups and reducing the load on operational servers, but it's not a free lunch.

Reply on news.ycombinator.com