Our data-heavy API returns JSON responses from our DB in ~40ms using ActiveRecord. So no, it doesn't waste 100ms on every request! We're serving around 6000 requests like that per second.
The Heroku dynos I've tested with some years ago performed quite horribly. Only a proper Xeon server was able to achieve sub-100ms responses.
But hey, if Ruby improved in the meantime, cool.
I've always thought of it as the platform you use for running toy apps with maybe a couple of dozen users at most and a low load. It's incredibly quick and easy to get a Rails app running on Heroku, a couple of hours at most. But you pay (in dollars and performance) for this.
(Then again, they have them with Phoenix as well but to a lesser extent.)