Colloquial usage around here means something close to, "An inability to gracefully accept an external perception of ourselves, because that perception is at odds with our own understanding of ourselves."
This "fragility" must always be accompanied by an accusation - otherwise fragility would not manifest. Without an accusation to deny, no one can be fragile in this sense. The opposite of fragility is callousness - i.e. "Damn right, that's what I said. I meant it. I don't care who it hurt, as long as they get out of the way."
The word "fragility", in its current social context, would appear to be a word intended to belittle. It says, "You think you're a big person, but you are a small person. Because you have denied my truth, you are fragile. Because you have proclaimed your innocence, you are fragile."
To me, there is a glaring fallacy in calling people "fragile" this way - and that is the part when the people who engage in this behavior decide to forgo intelligent discourse and presume that they are correct without hearing the other side of the story. Sometimes people are different than each other, and you can't always win arguments just by calling them "fragile" when they disagree with you, because what we're really talking about sometimes when we say the word "fragile" in this context, is "an impassioned personal defense against accusations thrown at people who look like me, but who do not represent my values." It's not okay to do this. The only thing we do when we call people "fragile" in this way, is we internally invalidate their position, and then we burn the only bridge we ever had with them.