Well, that, and there's an orders-of-magnitude difference in homicides between trans and cis people. (And, of course, it's worst for young, black women). Being openly labelled as transgender significantly increases your risk of being assaulted.

That's why misgendering is so insidious - it's stochastic terrorism. The person doing the misgendering can claim that they never meant any violence, but they're still increasing the chance of violence.