The difference between the two parties is information asymmetry, not intelligence.
Walmart is organized and concentrated and can bring to bear entire departments of lawyers and analysts. It gets to repeat the transaction N times and amortize its costs across all the transactions.
The individual workers do not have such resources and are not unified. Each of them is at a severe disadvantage in the negotiation.
The result is an inefficient labor market.