I found this annoying at first along with not being able to save files to the server. However, Heroku is forcing you to automate server setup so servers can be restarted, destroyed and created without losing data or functionality. If you allow SSHing in to install software and storing files on the server, you're probably going to get some nasty surprises when you need to recreate the server or add more servers for example.
You mention it was a couple of months of work to transition to your EC2+Docker setup. How much time do you find is required to maintain this compared to your Heroku setup? Heroku does seem expensive but I find it can have lower setup and maintenance costs than a custom solution.
Being able to run and test your Docker setup locally is a nice benefit though. I haven't found a satisfying solution to this for Heroku yet.