Using the same reasoning to outlaw prostitution is harder. Sure you could argue people have a right to not be a prostitute -- and I heartily agree with that -- but then the job of the law is to prevent people from being prostitutes against their will. All outlawing prostitution outright does is hide it from view. It doesn't help people get the rights they deserve. In fact it takes away the right to be a prostitute if you want to. The only line of arguing that prostitution should be illegal is either because we should outlaw anything we deem immoral, or because somehow concocted a right that prevents people from doing what they want to do.
In my opinion the basis for any system of law ought to be that people can do what they want to. The only exceptions being when doing so infringes on the right of others. I should be free to be a prostitute. I should not be free to force other people to be prostitutes against their will. I should be free to wildly swing my arms. I should not be free to punch someone in the face.
Of course everyone has their own opinion of what the law should be and so we end up with a patchwork of intentions -- let alone how the law actually gets applied. That's fine. People are free to have opinions.
Either laws are written based on morality, or laws are written based on rights. You can't have both without being inconsistent.