The moral justification in my mind is simple: morality is a universal social contract between all beings capable of reciprocating moral obligations; sheep aren't capable of reciprocating moral obligations, hence we have no moral obligations to them.

This isn't a popular viewpoint, and it's counterintuitive to people who have been raised to be affectionate towards animals (myself included), but you asked for a moral justification, not an emotional justification.