Yes, it is trivial to run your own email, storage, backups, etc. In fact, you can't even use amazon for email since you can't set up a ptr with them.

But if you do want to use Amazon securely I believe that they have the ability to encrypt your drive.

That said, rack space is incredibly cheap these days, and throwing in a 1u ubuntu server in a 1/4 secure rack is cheaper than hosting a dedicated vps at amazon.

I completly understand starting out using shared services as a startup, but eventually, and soon, you'll want to move to your own systems. It's just an expense for your business, and you would be irresponsible to leave your systems vulnerable.

(my projects have been more sensitive in the past than the typical CRUD app, and so perhaps I'm paranoid)