I just use a VPS, using Ansible to automate setup of dependency and Capistrano to deploy app (Ruby on Rails), very happy with this stack.
Have you reach to the point of having to use a load ballancer? If yes, could you tell about how, please?
There may be better options out there, but it will probably take a fair amount of time to outlive it. (assuming you're not a rocket ship unicorn)