I can see how systematically denying someone services is harm, but refusal to engage with someone as an individual? I'm not convinced. At the end of the day you're compelling someone to perform work they don't want to do.
And I understand the "rules of our society" stuff, but I'm trying to get at it on principle. And I am not saying commerce should happen with no rules, that's a different discussion, I'm talking about this specific set of rules.