Any help would be appreciated on EFnet IRC channel #firespring or #archiveteam
Archive team projects: http://archivingyoursh.it/Current_Projects
I'd love the idea of trying to take over something like this, but I have a suspicion I might be in way over my head...
- Our cassandra and Redis clusters are/were the most expensive pieces of the architecture.
- the PHP front-end wasn’t really efficient and required many `medium sized` instances.
- Sites with huge fanout effects are expensive to run.
- Multi-platform : iOS, Android, "touch" and www. Maintenance gets expensive.
- Multi-AZ on RDS comes at a price
- If you intend to run on AWS, Reserved Instances are your best friends.
Formspring reached 1 million users in 45 days. How do you plan for this on typical hardware? Especially when you had no idea ANYONE would use your little side project?
AWS isn't cheap. It offers building blocks upon which you can "easily" build and scale a website. Not for everyone, but good enough for us.
In short, yes. This is inherent to the nature of Formspring – or to some extent social networks.
Since most of your actions on the site are being `broadcasted` to all your followers, and since the distribution of followers follows a pareto distribution, it is very difficult to predict load at any given time, as one single action by a very popular user could trigger hundreds of millions of events (publishing to everybody's stream, sending emails, push notifications, updating counters, spam filtering, etc.)
We benefit greatly from the elasticity of AWS, and `scale up` accordingly to deliver this event in a timely manner.
If we were not concerned by `delivery time`, we would be able to smooth it out, but at the expense of engagement… and as you can imagine, engagement in a social network is quite important :)
Many of us are not even playing the same game.