Even if it were an easily changeable characteristic, we should consider it wrong for any society to threaten harm or punishment or ostracization for this. That the characteristic is or isn’t easily changed is not relevant.

It’s not a question of moral relativism. You’d have to argue that any behavior is a crime in order for it to be legitimate for perpetrators to suffer consequences. Otherwise, it is wrong that someone should suffer consequences, all else equal.

I like nachos. Tomorrow I might not like nachos. It is wrong in any society if my liking or disliking of nachos results in me being killed, bullied, shamed, etc. Any society that claims a right of legitimacy to impose social norms in this way on the liking or disliking of nachos is a morally wrong society. There’s no debate about this. You can either argue convincingly that nacho preference is a crime, or else any attempt to treat it like a crime is wrong.