I find Linode 512MB hosts more than enough for development need, and the automatic backups make me feel better about leaving unfinished code and scripts on my server.
Also, if you gain a team member, you can easily clone your VPS to make a new development environment for the team member.
With all that being said, I use and encourage all my employers heading to the cloud to use AWS for production servers.
On the other hand, after launch, AWS is amazing to scale with. I use AWS in conjunction with Rightscale which I highly recommend. Much easier to deploy.
I think you could do something similar with Slicehost/Linode using disk images and configuration profiles, but it doesn't seem they offer anything like Amazon's machine images. Correct me if I'm wrong, though.
Of course, the flip-side of all this is that any individual instance of an AMI is inherently ephemeral, and you need to treat it as such. Even the reliability claims of EBS are fairly weak. [1] That means EC2 instance storage, even EBS storage attached to an instance, isn't truly persistent, and I haven't quite ironed out my system for dealing with this.
You'll probably want to run your own Git server [2] and commit code as it's completed. That still leaves the possibility of data loss if your instance dies after you've written code, but before you've made a commit. You could just take periodic snapshots of the EBS volume where your in-progress code resides, but whatever period you choose for the frequency (like every 30 minutes) is equal to the amount of work you could lose. And that sort of constant backup isn't really what snapshots are meant for anyway. [3] I think the ideal solution would be to run Dropbox to get continuous backups of the files you're working on. That might also make recovery easier. Attaching an orphaned EBS volume to another instance isn't hard, but it's certainly not as easy has having Dropbox download the file you were just working on.
--------------------
1. http://blog.rightscale.com/2008/08/20/amazon-ebs-explained/
2. http://alestic.com/alestic-git/ and make sure you're taking frequent EBS snapshots. The right way to do this is with ec2-consistent-snapshot and XFS (see http://alestic.com/2011/02/ec2-consistent-snapshot-sceencast).
3. I might be wrong about this. The fact that you should lock the file system to ensure a consistent snapshot makes very frequent (say, every minute) snapshots seem like a bad idea. Maybe you could get around this by not locking the file system for snapshot when dealing with plain files (and not a database), as consistency should be less of issue. Maybe I just don't know what I'm talking about ;-)