We're currently running most of our stuff from a Mac Mini we own (about US$1500) that's co-located with macminicolo.com in Las Vegas (about US$30/mo). It's mostly a static sight right now, though we'll be setting up an online store to do direct e-book sales in a few weeks. We will also get a second Mac Mini sometime soon to use as a staging/emergency replacement server.

We're selling e-books to a niche market (check out www.photoshopdimensions.com) so we're not worried about runaway traffic growth, though of course we'd love to have that problem.

This has worked out well for us so far. Setting aside our questionable server OS choice, the downside of owning your server is the higher up-front cost and responsibility for the hardware. You can reduce this by leasing a physical server from a hosting company.

The upside is you have the full server CPUs/memory/disk at your disposal; a modern server can support a pretty sizable load if you don't have to share it. I believe StackOverflow got pretty far in their early days on a single app server and a single DB server.