Just because it is in the constitution doesn't make it right. That is a document designed to be changed, after all.
Women couldn't vote for quite some time. Some people weren't fully human. Only landowners could vote. And so on.
Furthermore, it being in the constitution still doesn't detract from it being slavery.