It just seems surprising to many of us that this skepticism only seems to crop up in situations surrounding racial or gender discrimination.
The more serious/terrible the accusation, the more likely it seems to me that people will just take it on face value.