Does that work with s3 static websites though? I recall that a different endpoint was needed, and it's only http.
CloudFront has its own IAM user that is permitted to access the contents of the bucket, which means that it has to use the API. I don't think it's even possible to access the S3 API without using HTTPS. Therefore I think it is highly unlikely that the connection is unencrypted.