Why don't you tell me. One of these is female (and pretty accomplished in the open source community), one of these is non-white ... which is which ?
* thegrendel.abs
* tille
* pgquilles
If you can't, obviously the reason is not just racism. On mailinglists, github and irc, skin color and sex don't exist. If there is a difference, MAYBE it's the result of companies not hiring these people to work on open source, or because sex and ethnicity influences whether and how much you work on open source (actually I think it's more about location. Open source contributors almost always live close to universities that were very involved in the creation of open source 20-30 years ago. Because those universities are effectively only in USA and northwest Europe, the ethnicity is skewed, but there are plenty of mostly white countries that are massively underrepresented in open source projects).
As to why people are hostile to other groups, a very significant portion of them are either teenagers in highschool or university students (and even then, skewed towards the younger univ students). They're not a welcoming group, no, that's true. But they are pretty universally not welcoming, asking for someone's sex/color on lkml ... is not something you want to do if you want your reputation to survive. Hell, there are flamewars that started because of statements that someone was somewhat richer in kilograms than medically advisable, when it was absolutely true.