The thirteenth amendment to the US constitution is also restrictive. It protects the right of people to not be owned, at the expense of our right to own others.
Not every possible action is a freedom. There is no right to own others in any cohesive ethical framework. (It would violate the principle that all humans ought to have the same rights.)
Why is that necessarily a principle that has to hold for an ethical framework to be "cohesive"?
Because it otherwise implies that one group of humans has different rights than another group of humans.
I'm sure such a framework could be constructed, I just wouldn't call it cohesive.
I think we need to unpack "cohesive" to meaningfully communicate further, here...