I think the basic concept to understand is that of the 'dyno'. Although a term from the Heroku platform, I believe it applies to most of the other platforms too. A dyno is a container (sort of like a VM but without hardware virtualisation, see LXC or OpenVZ) that isolates system resources. You can have multiple dynos on any given server and you can have lots of servers. Each dyno contains the entire environment needed by any given app, so it'll contain the specific binary version of ruby or node for example and it'll also have unique copies of all the dependencies, managed via bundler or npm for example.

Scaling your app simply means having more running dynos that are load balanced by a centralised load balancer used across the entire platform. So yes, interestingly there is a lot of duplication of the code base.

Services, like databases, are managed on their own separate servers/clusters and allow multiple connections and can be scaled/sharded separately from the dyno farms. Therefore they can be monitored, maintained, upgraded, etc, separately as well.

I'm a small-time contributor to http://github.com/openruko, an opensource Heroku clone.