I think this issue is vexing because there are so many groups with different compelling reasons to favor or oppose prohibition. The below list is obviously a gross over-simplification, but I think it captures the largest forces in the debate.
FOR: Men. Men would benefit from legalization in the obvious way.
AGAINST: Women. Women stand to lose if their partners are tempted by legal prostitution.
FOR: Sex workers. Sex workers would benefit in the obvious way.
AGAINST: Religious groups. They lost the battle against pornography, but they're winning this one.
AGAINST: Feminists. Not all feminists, but many feminists believe sex work is exploitation by definition. [2]
Each group has powerful motivations completely independent from the effect of legalization on sex slavery.
1. https://www.forbes.com/sites/timworstall/2013/06/15/legal-pr...
2. https://en.wikipedia.org/wiki/Feminist_views_on_prostitution