I'd be pretty careful about this line of argument. What we think of as a clue to underlying qualities is cultural... for example, one of the biggest superficial differences in people is the color of their skin. What underlying qualities does skin color clue us in to?
In the US and Europe, we have a long history of trying to find the ties between visible and non-visible qualities. Phrenology, eugenics, Blacks as having "inferior intelligence", the belief that women are prone to hysteria - all were attempts to find that link.