Good point. It's actually generally cheaper to serve content through Cloudfront instead of S3 directly as bandwidth is less expensive with Cloudfront.

However you are going to pay for Cloudfront requests in addition to S3's. In the vast majority of cases, it's insignificant compared to the savings you can achieve on the bandwidth side. (And at a certain level you can ask AWS to waive CF request fees entirely).