Maybe someone in your field would know, is there any difference in the actual reliability/reproducibility difference in research done by men and women? Maybe you are simply measuring a real phenomenon but in a round about way?
I think if it had been a larger study (not a one credit class run by 4 undergrads but an actual study with funding) we might have had multiple texts and randomized them, but we had one text- I don't even remember what the topic was.