Here's the writeup: https://www.appneta.com/2013/02/21/the-taming-of-the-queue-m...
(Spoilers: turns out it gets a lot better very quickly.)
The solution is to have dynos large enough that you can run 8+ workers and let the operating system do all the complex scheduling.
So Heroku could spawn workers with a $FD environment variable instead of $PORT and the "complex scheduling" done by the OS _is_ the second routing layer.
But really, they could still do a second level routing even outside a single OS as the scale of the distribution is much smaller, so having the routing mesh be aware of the worker availability seems feasible again.