It's also not wrong to say 'maybe we should try to make our company more representative of society'.
>> But if the difference is due to expressed preference, its fine and we should all chill out.
The problem is that systemic prejudice causes long-term harm to human society. Women whom make less simply because of their gender have access to fewer opportunities. So do their children. They retire with less. It goes on and on...
For many people this issue isn't something that can be summed up as simply as perhaps you would like. It has real, long term consequences.