There is a fundamental moral difference between using violent threats and physical confinement from using offers of payment to compel work.
Taking a job because the alternative is to starve is still a consensual interaction. We need to work to survive. We can go out into the wilderness, and live off the land, which would let us survive at subsistence levels, or we can go into civilization and work within an employment contract that gives us a much higher standard of living.
The freedom to choose ensures that generally speaking, people in a market economy are choosing one of the better options available to them, rather than one of the poorer ones. It also ensures that when people DO interact with others, it is because it is generally mutually beneficial. This basic dynamic leads to an emergent property of actions carried out in markets generally being a net-value gain for the economy as a whole.