I've worked in small teams where I was the only Asian person among a team of white men, and I neither noticed any hints of racism nor sexism.
I've also worked in teams where it's been highly diverse in ethnicity though not gender, and again, never noticed any hints of sexism.
Though I have certainly worked in a company with a number of women, including a female co-founder/CEO who was absolutely sexist... against women. I'm not suggesting women are bad CEO's or that diversity is bad, I'm saying that from my experience, it hasn't been about the diversity of the team so much as the type of people in the composition.
Also from my POV, I'm not sure how having a random % of women makes more or less sense than a random % of black people or Latino's. Like if a company is predominantly composed of white men, does adding a white female make it more diverse than adding a black man?