Designing a Scalable Twitter
natishalom.typepad.com
natishalom.typepad.com
Otherwise a good read.
There are different levels of scaling. The more users you have, the harder it becomes. But it's generally not worth scaling beyond what you actually need because if you don't need it, then you've wasted tons and tons of man power that could have been invested in other areas.
Speed has nothing to do with it.
A simple stateless website that requires no shared data like from a db, is scalable, since you can add more web servers to fulfill more requests.
Using memcached, however, isn't scaling, because while memcached itself does scale, if the backend it reads from doesn't, the whole system doesn't.
Sure, the latter architecture can serve 1 concurrent user, but it's also 100000 times more expensive. The answer is not as simple as "it scales or it does not".