Possible reason being, the more likely you are able to socially connect with your potential victim and the less likely you are to see them as the "other," then the less likely you are to feel justified in causing them harm.
KKK feels justified because other races are inferior, the rich feel justified in exploiting the poor because poor people are lazy, e.g.
Like you say, it's not hard to believe that homogeneity (whether race, religion, culture, ethics, politics etc.) causes less conflict. Naturally, that's an awful justification for a terrible practice.
But you find many tunnel-visioned "the statistics justify the means" arguments: minimum wage, gun control, privacy, etc. Proper laws are not just about statistically good outcomes.