I began my life at a startup 1 year after launch. 3 Servers, 1 web server, 2 app/db servers. As load increased and our needs changed we moved to a more complex structure, 2 load balancers with failover, 2 web servers and still 2 app/db servers but beefier. When life got harder we started optimizing web servers software and the database. When traffic grew past that along came another server cluster scaling us to an even greater degree.
Do we deal with hardware ourselves? No, managed hosting with dedicated servers. Why? Because we want to let someone else deal with the hardware problems. The cloud doesnt have the resources we need. We scale when we HAVE to, as in the site is going down every day because we cant handle the traffic. We run leaner than some of our competitors and we know this because some of them have gone public with their costs, number of developers and server architecture.
1. Have a plan 2. Find a hosting company you can scale with easily 3. Use the minimal resources necessary 4. When you have to scale determine the cost of optimization vs just adding more hardware. 5. Be Lean