The numbers I've seen are opposite to what you claim, and seemed to indicate that a lot of the "trafficking" is actually people traveling to another country for a while to earn (relatively) easy money without risking getting stigmatized by people they know.
You can see sort of the same thing play out without the stigmatization with carpenters and similar coming from low-wage countries. They'll generally have worse working conditions than local carpenters (if for nothing else being away from their families), but still come for the opportunities.
I do think the stigmatization makes it a difficult subject.