While RapGenius it right to point out that Heroku was incorrect in describing how their load balancing works. It seems off to blame this exclusively for their performance problems. Plenty of high traffic sites not on Heroku operate just fine using nginx's upstream load balancing, which is simple round robin load balancing, paying no attention to how many requests are being handled currently by a backend.

What seems particularly odd is that rap genius appears to be using their Heroku dynos to serve their css, javascript and certain images. So loading the homepage appears to make about 12 requests that hit their dynos, rather than 1.

If you were looking for low hanging fruit to reduce load on the dinos for a high traffic site, this seems like an obvious place to start, rather than jumping straight to a 'smart' load balancing solution.