My answer comes from the perspective of a recent grad who has spent a year at a mid-sized SF social gaming company, working only recently on the back-end (i.e., not a scaling expert).

A few things I've learned w/respect to scaling in my context:

- I/O is likely to be your bottleneck, so design your db well and anticipate splitting it across multiple machines (and what that means for your access logic)

- don't spawn a new thread for every request (a la apache)

- keep your services simple (perhaps: one for handling web requests, one for data access, one for caching, one for your payment system) and their relationships even simpler (one hop max from RPC caller to callee)

- cache like a hoarder

- find/write an efficient serializer for RPCs between services

What helped me get a grip on this stuff was sitting down with an architect who has done it successfully multiple times. I asked for an elevator pitch description of scaled web architectures, and then ask him about his failures. Voila bullet points.