I use a $6/mo box for my primary business hosting, but I have a $3/mo one that I'm using to build v2, really just to prove what's possible. If you set up your DB and caching right you can do so much with so little...
Yeah - I remember a StackOverflow talk[1] where they basically said that they just vertically scale their database.
The fact that they were able to make it work tells me that most businesses should probably just go that route and avoid the headache of distributed systems[2].
---
1. https://www.infoq.com/presentations/stack-exchange/
2. Obviously a business should probably invest in redundancy when it comes to data (as did StackOverflow), but a pure "Raid 1" setup is the easiest of distributed systems to understand.
You get a lot of bang for your buck out of them.