Basically, if you take the axiom that physical appearance and genetic qualities should not matter at all, then we need to minimise its' influence.
But you could also easily make the argument that beauty is a proxy for a lot of things that we _do_ want to maximise. It makes sense to reward those that are beautiful, fertile, strong, etc.
Just not above all else. You wouldn't pick a romantic partner, or a friend, or a colleague _solely_ on those qualities, but to not incorporate them at all seems odd.
I mean, on a basic level, you have to be able to get on with your colleagues. A lot of people would find it difficult to get on with someone who's ugly, doesn't shower, has no social skills, etc, even if their code is great and their stock picks too. It's not arbitrary.