Yes, but these actions need not the same response as others that were meant to be harmful. There's a difference between murder and homicide, for example.
To bring it back to the question of whether it's appropriate to respond to unwitting acts of sexism with anger: Depending on the particulars, it very well can be. Calmly talking about things is preferable in many circumstances where civility is valued at a premium, but it has limited utility in that it isn't particularly salient. Whereas a few harsh words can really grab someone's attention, and can show them that what they said was hurtful, rather than merely telling them. And, this being the 21st century, oftentimes the reason why someone might still unwittingly ostracize women is quite likely to be because, to put it bluntly, they're kind of dense. Meaning that harsh words might get through to them where soft words won't.
Done correctly, it can still be a compassionate act. Especially if the attention-grabbing jab is followed up by more patient, civil conversation. A little pain now in exchange for a lifetime of improved relations with the rest of one's own species is a pretty good deal, all told.