Yes with a CNAME and some metadata on your S3 buckets. See here http://aws.typepad.com/aws/2011/02/host-your-static-website-...
Now, if you're serving large files and you really need more than 100Mbit sustained, S3 makes sense. But it's unquestionably a premium service for a premium price.
Now, when people say that S3 scales well, I totally agree. But why do they say that the prices are competitive, that's beyond me. Take Xirra's XS-12 storage (200€/month for 36 TB) and compare it to storing 18 TB (I assume RAID 1) on S3. 1 TB costs $95 on S3! How on Earth can you call it 'competitive'? Now, that's even without bandwidth costs. I totally agree it's a premium service for a premium price. There are plenty of cases when using S3 is just a big waste of money. (And Amazon's decision not to implement cost capping isn't helping either.)