If the model statistically significantly returns a different number for men and for women when controlling for all other factors, then it's a bias. I don't understand how this is even contentious.