I agree with this statement. Having or not having a CoC will not change their behavior, nor should it change your response. But the response to your response can differ: If you've been transparent from the start about your rules, there's less room to complain about you enforcing your rules. If you had no CoC beforehand, there will be some parts of the audience asking if that was really fair. And this behavior doesn't even have to be malicious, just that their line is somewhere else, and so they don't understand your arbitrary enforcement. With clear and transparent rules, they may not agree, but it's clear where the line is. We both think (at least, that's what I gather from your comment) that some rule is necessary, but I'm for being open about it and you want to keep them vague. I think it's easier for everybody to be transparent here, because you can decide beforehand if you agree to these rules. If they're too vague, I'm just hoping that, if push comes to shove, we see the world similarly.
And, again, having a CoC also signals that there is a line. Without it, who's to know that there will be behavior that you don't accept and what it looks like. Maybe you're fine with sexist comments, but not with racism. If I'm vulnerable, why should I gamble that your views are similar to mine? I'd rather not visit your conference if you can't make the rules clear.