Heroku is a god send for getting off the ground, however the biggest issue for us hasn’t been cost (it’s high, but worth every cent) it’s been server location.
We’re an Aussie company, and a large percentage of our customers are in Australia. Having servers only in the US and Europe mean that Australian customers speeds are slower than they should be, and private spaces are far too expensive to warrant it.