Scale Cheaply - Sharding
codebetter.com
codebetter.com
This doesn't apply as neatly to something like Twitter. There, we have all our users potentially needing to be joined to data on all our other users. Joins don't go across shards. You loose that with such a system. So, for something like Twitter, you can either run lots of batch jobs and deal with the latency that batching implies (not an option for a service that is used for instant, small updates), use a single DB (replicated still applies as single in this case), potentially use something like memcached to store a giant hash table of the relationships and updates for the past 24 hours (and the database only becomes long-term storage).
Sharding is awesome. Sharding has great uses. Sharding isn't something most of us will have to use.
I've been trying to think of ways to speed up the app I'm working on. The current bottleneck is in the database queries, sometimes they can take up to 30 seconds. The data is mostly static, it's updated once per day.
Making users wait this long sucks. :(
So I'm thinking perhaps a combination of shading and de-normalisation could really help out.