1. At any scale pretty much AWS / GCE are less expensive than Heroku. They also come at an non-zero operational cost. Say, switching to Google App Engine (the cheapest from my understanding) -- would save them 75% of their bill (just back of the napkining). -- That's not much, compared to the cost of having someone migrate the platform over (multiple work-weeks)
2. S3 != Imgix. Imgix is a CDN + Image resizing service. At a couple employers ago we tried to run our own image resizing service in house, and it was a royal pain in the ass. I can go into this if you're interested.
The other thing is S3. My current employer uses S3 pretty heavily, and S3 is really not meant for end-user data access it seems. It suffers from lots of latency spikes, and a non-zero error rate. Slow, and unreliable sites make your users go away.