I can provision and configure a dedicated Ubuntu machine at Rackspace Cloud with all those services in about an hour (and I am far from being a sysadmin guru, I just follow the wiki instructions and best practices for security/firewall etc), and it costs me $12/month.
Imagine a world where your application has enough traction that you're spending more of your time fixing issues or adding features to please/retain the maximum number of users.
Having been there, when that day comes, you won't be nickel-and-diming. You'll just want some rock-solid guys in your corner who allow you to worry about one less thing.
When my app needs to start scaling up a lot (and that's often an "if" not a "when" for a lot of projects), then for sure I agree that a well-managed service backed by talented people is worth every penny.
I think if pushed on it, most developers would prefer to spend their time writing code than setting up and configuring servers. Which is why we created Orchestra.
Each to their own though :)