It's not about difficulty, it's about time and money.
I _can_ manage a server. But for $25/month to just forget about it? I'll take that tradeoff any time. (And have, back when I had production machines myself. And will, if I have time for that side project again sometime soon...)
And for stuff that needs more than one dyno, well, if we assume $100k/year for an ops person, that still gives you sixteen of Heroku's beefiest instances for the same price? I still think it makes sense.