For static websites, Github Pages is free, nearlyfreespeech is very cheap. For full-blown web apps, didn't you hear the guy? GAE has a pretty generous free tier to test out stuff. So does Heroku, and it's way more user friendly. I still prefer GAE because I can use Google's tools. If you want to rent instances, you're going to have to pay from the beginning afaik. AWS and GCE are the usual suspects here.

https://pages.github.com/

https://www.nearlyfreespeech.net/

https://developers.google.com/appengine/

http://heroku.com/

I'm using the free tier of openshift from red hat.

The performance is good enough that I'm able to cram a java application powered by jetty + mysql + phpmyadmin in a single small "gear" (they give you 3 small gears free, 1gb of hard disk).

AppHarbor (a YC company) is nice for .NET stuff. It's free unless you want a custom domain, in which case it's only $10/month.

Microsoft Azure is pretty easy to use and has continuous deployment available from Git repos. There is a free tier where you can have 10 websites.

I've got an MSDN subscription, so I've got free credits to bring me to a higher tier and it's a no-brainer. You could also qualify for the BizSpark program which provides you with a free MSDN subscription.

If you're willing to get your hands "dirty", you can self-host on a vps, it's damn cheap those days. DigitalOcean is a favorite around here, but there are also more "professional" companies like Linode, and more cheap options like my favorite Ramnode.