Assuming perfectly spherical cows..
Actually, I agree with you, being the sysadmin you mention. Unfortunately, the up-front cost seems to be a barrier to many a startup founder. To me, this suggests a lack of good leasing options, though it could be that's because it's not worth making a business out of it.
I understand you don't have to worry about hardware issues
I've always found this assertion to be misleading at best. One still suffers the consequences of hardware failure (both inherent and human error), yet one has very little, if any, ability to prevent them. You're at the mercy of the providers.
It just doesn't make sense outside of well funded startups or established businesses.
I'm not sure it makes sense inside them, either. I've found that the break-even point for buying versus cloud is consistently around 10 months. The rule of thumb for when the transition from cloud to colo makes sense for a startup that doesn't yet have any of their own hardware is about $10k/mo, assuming they're growing at a rapid clip.
The only situation where it would make sense is compute-heavy (but light on memory and I/O) loads that peak much higher than the average. I have yet to encounter such a one, since it's tough to meet the parenthetical requirement, thereby bypassing the Deutsch distributed computing fallacies.
That is, 3.7 month break-even is a much stronger argument than 10 months, so I'd be interested in hearing what others have found to be the break-even point with various cloud providers.
Once you have a single project that justifies the colo box, the marginal cost of starting a new project with a new website is zero. No need to configure a $20/month slicehost box and cross your fingers that it can handle a Redditting. You just put it on the monster box that can handle anything you throw at it until it demonstrates that it needs a big machine of its own.
Colocation is still useful in startups if you want to create/delete instances all the time in a development type environment where you're testing out the infrastructure that integrates between VMs.
Are you using Xen to virtualize a bunch of servers on it?
I've always figured colocation + System Admin ends up costing more than a cloud solution. Especially when you need to have enough redundancy so that you don't fall over if a single server fails. In the cloud if a single server fails, it's their problem. In physical space it's your problem or your system admin's problem. How much time can you spare?
I am not using Xen. I can see the benefit for some stuff, but I don't believe it's beneficial for my use case. However, I will make that decision in a more informed manner when I am actually in a position to expand our hardware (money and need).
I don't think the colocation costs more. Seriously, I have had great use out of my current server. I figure with a colocated set of 4 servers, $10k initial expenditure + $300 a month for 2 years is $17200, or I can go with rackspace and pay $3000 a month for 2 years, and be $72,000 in the hole. Oh, and after that two years, I don't even have spare hardware to play with!
Most startups fail. You should be prepared to fail cheaply. You can always spend $50K on servers once you are successful. Let's say you spend $1K/month on cloud-based servers. That's quite a lot of horsepower. If you fail in 6 months, you are still $4K less broken that you would be had you spent $10K in the first place.
If you succeed, you will able to spend as much as you need in servers to keep your bases covered and still add and remove cloud-based servers to lessen your load on demand.
The advantage to 'cloud' stuff is that if you have servers with less than 32GiB ram, the economics of owning get much, much worse, mostly due to hosting costs. If you have tiny little servers, VPSs really are cheaper over the long term... of course, 'serious business' hosting usually requires larger servers, so we're back to saying that you save a lot of money buying if you are big.