To extend your analogy, if I may, Heroku is like an ORM which charges per month, and where it costs more to use relational finders. You're paying extra for each convenience and giving up some control, and this is an ongoing cost. For some people this is a great fit and they really don't want to know about this stuff, are building a demo, or working on a smaller site where you don't want to have to babysit the server and are willing to pay a little extra for that.

For others who are on a tight budget, or have an ops person or willing devs who would prefer to control the server setup, heroku is not such a good idea. I think people overestimate how hard it is to get a typical stack up and running and keep it up - for a small server or vps (i.e. most startups) it's probably only a few hours a month work plus a few hours learning setup, if that. The second one you set up can be up in minutes, and it gives you greater control in the long term, plus obviously it's cheaper. If the amount you pay heroku is less than a few hours of your time, then why not use heroku? However if you grow and are spending a lot of money with them, you've probably outgrown them IMHO. I do like their service, and have tried it out, but the convenience comes at a cost, which increases rapidly the busier your site is.