Does VPS have any advantages over other services? I'd refrain from using it--no git, not scalable, maintenance required.
The advantage, if you are into that sort of thing (managing servers), is that it's fully customizable however you desire. You can most definitely install git on a VPS. As a side-effect, you'll learn some server management/maintenance things.