It makes less open-minded people uncomfortable, to be around others who are not like them. This is what I mean by 'unfortunate': that not everyone is comfortable in the face of different.
If you bit into a steak and found that it tasted like chocolate, you'd be confused and disturbed. Now multiply that feeling by a large factor.
The only reason to be possessive about someone else's sexuality is insecurity, I think. Which can be very visceral, too, and loves to camouflage itself.
I (obviously) support equal rights and access to life saving hormones and surgery for transsexual people - since I am one myself. Yet, from my elementary knowledge of evolutionary biology, it's long been my suspicion that there is some sort of deeply rooted antipathy towards trans people that is not completely socially constructed. After all, having sex with a trans (pre or post op) person in modern times, or an intersex person (modern or historical times) would seem to work against propagation of someone's genes.
Still, the bigger causal factors in transphobic sentiments are, I believe, socially constructed and rooted in patriarchal misogyny. There are way more (cis) men with violent, transphobic attitudes against trans women than there are (cis) women with violent, transphobic attitudes against trans men.
Perhaps more relevantly, it makes less open-minded people uncomfortable to be around others who don't fit their mental model of the world.