I, however, don't know very much about being a sys admin and I certainly wouldn't want to be put in charge of maintaining any production server.
That's what DotCloud is for. I have a beta account with them and its been working out great so far.
I've told a few people that we run on Heroku and had them respond "Really? Isn't that expensive?" When I explain that we have a whole bag of add-ons, several dynos, workers, etc, and our 6 months of expenses are still roughly a week's worth of man hours, they realize that it's actually quite inexpensive.
Seriously, I'm making a large part of my living off companies that once thought (or were told) that they could do without a sysadmin.
If the internet-part of your business consists of more than a static website (or a wordpress) then, by all means, pay that fulltime salary. Even if you decide to go fully managed hosting; you absolutely want that to be an educated decision rather than an educated guess.
And it doesn't end there. A never-ending chain of small decisions is waiting on every step of the road. Pull that knowledge inhouse (or to a trusted longterm advisor) as early as you can. Otherwise you will inevitably foot a much higher bill later in the game.