You certainly can burn through ridiculous amount of money if you insist on purchasing servers outright and pick expensiv brands.
Conversely, the "private cloud" deployment I run are mostly rented servers with month-to-month contracts. No upfront infrastructure spend - we grow them a server at a time. We also run some bare metal setups and they're generally cost-competitive with rented servers, but there it certainly depends on the size of the installation.