Well, normally morality is "decided" by a social process, which also shapes the rules, so they don't often diverge by much. But my point is to separate the two issues: society decides the rules; who "decides" morality is another discussion.
Any freedom is some compromise. If a society has two people or more, then either one person is allowed to, say, enslave the other, in which case the society isn't totally free, or not, in which case the society also isn't totally free. So there is no such thing as absolute freedom, and whenever we say freedom we actually mean some point on a spectrum. We could argue over what that reasonable point is, but absolute freedom is something that can't exist. So instead of speaking in absolutes, let's acknowledge that we're arguing over a favorite compromise.