That sounds all reasonable and stuff. But it's like saying if you can't pee in a urinal, get out. Go somewhere else.
The fact is that women get harassed far more than men. Taking away blocking will negatively affect them disproportionately. Ignoring that isn't being "evenhanded." It's being willfully ignorant of discrimination.