We use Heroku heavily, and we went from 2 full time devops engineers to 0. Everything is now buttons and sliders. There are no "security patches". The CEO could log in and scale if he needed to; it's a slider. We get monitoring for free (labour free, not cost, i.e. the "good" free). Memory usage, CPU usage, HTTP status codes, logging: it's all there. We spend no time thinking about rsync or devops or any of that: we just solve the problems we're good at.
Of course, everything is a matter of scale. Legend has it, Deliveroo UK only moved off Heroku after they grew so large, Heroku wasn't willing to offer them more dynos on their account. That sounds like a reasonable time to go in house. But any <100 people company.. why bother? focus on what you're good at, and let other people do devops.