For me, it's a matter of being misunderstood by everyone. Literally. I don't mean they can't parse what I say, I speak English perfectly clearly. It's that they ascribe thoughts, emotions and motives to my words and actions that are wrong. They then react to those misunderstandings and I find myself in a constant state of defense and trying to correct misperceptions, usually in futility. After the frustration and anger it definitely leads to an enduring, moderate sadness.
Middle aged white men are not all cut from the same cloth, any more than any other group is. For others to tell me what I think and feel based on my gender, race and age is as bigoted as anything else. Sad.