Two thoughts:
- It's not a ternary choice between being racist/bigoted, perfectly neutral, or practicing affirmative action. You (along with everyone else) have a set of implicit biases; to curb them and better yourself, you have to take into account things like gender, color or age.
- How do you not know these things in a small startup?