Now, imagine you enslave one of those groups for ~400 years, prevent them from voting or getting an equal education for another ~100+. Might differences emerge in how society treats that population?
If we defined the "groups" in a less historically informed way, we'd still have differences.
I think it's very easy to overstate how much those things are genuine differences in preference/ability. Allowing no-fault divorce dropped female suicide rates by 20%; were they happy in those marriages, or enduring them? Would they choose differently if offered the same opportunity?