"Why should we lessen or diminish the legal right of authors to control how copies of their work are distributed; the way we've been doing it works well."
is what you SHOULD have said. Doesn't sound as sexy, but here, the accuracy matters.
I don't think this should be discounted so quickly. Nothing in this domain — money, private property, laws, justice — are obviously naturally occurring things. One of the great traditions in the effort to understand how rights and value come to be is the creation of narratives that connect the rights to early natural states. We see this in Locke, Rousseau, Rawls, etc. You could just ask the question of whether the current state "works well", but then you are in the wheelhouse of Jeremy Bentham and other utilitarians. Fine, but so far these kinds of arguments don't lead to worlds many of us like.