Some people are like that, where if you respond to their snark, your response will be used against you to tarnish your reputation in the eyes of bystanders.
It is important to never let them get the moral high ground.
If someone says something like...
* "Your team is all white."
* "Your team is all male."
* "Your team is all American."
* "Your team is all able-bodied."
A better way to respond would be something like, "That's true. We're always looking for new talent and embrace diversity, just haven't had much luck yet. <hiring page URL here> if anyone's interested."
It addresses the problem, and contains an open invitation for people of diverse backgrounds to apply.
(All of this is assuming you're willing to work with people who aren't white, aren't male, aren't American, and/or aren't able-bodied, of course.)