The same grounds that we use to argue against slavery today. Just because the society around you doesn't believe certain rights are real doesn't mean you can't personally buy into them. "Rights are a societal construct" doesn't mean that nobody has a right to believe different things from society. It just means that our conception of rights derive from societal consensus, not a higher power or objective source, and conversely that there is no higher power or objective source we can invoke to override society's mores.
> If you are willing to recognize that society has had an incorrect view of rights in the past, you must recognize that there is a standard beyond society to appeal to.
No, you can just criticize them using the today's moral standards. You can be sufficiently committed to your moral standards to be willing to impose them on other people without simultaneously believing that those moral standards are a objective property of the natural universe.
Hume's is-ought guillotine is a thing. A normative statement does not magically become an positive fact just because that normative statement is an extremely strongly held (e.g. slavery is bad). The notion of inalienable rights is a rhetorical trick that lazily conflates the normative and the positive.