One problem is that S3's object availability is only 99.99%, which sounds good, but if your site has 20 assets it needs to load (html, js, img, fonts, whatever) then 1 out of every 50 non-cached loads could be expected to be missing an asset which can be a big problem for JS heavy sites.