I purchased an older HP Proliant server off eBay with 12 drives for just under $600 shipped for everything. I installed VMWare ESXi on the machine and then created 5 virtual machines. One machine for an email server, one for database server, one for web, and two staging machines for database/web server. Total cost for those machines was zero, and thanks to virtual appliances I can expand.
At that point I have redundant power supplies, raid hard drives, and plenty of backup swaps. I also can configure the server whenever to allocate resources as needed to the machines running. I can quickly copy and create a new server whenever I need one, which is really nifty.
In the end the only down-time I have is when my power is totally out. If I host I usually have about a 97% up-time excluding regular maintenance. The machine running constantly makes less impact than my Dell XPS 720.