So yes, we should improve it. This is one suggestion for how to do so. It costs more money, but we no longer have to make price the #1 objective, not in a world where people are willing to spend $4 and up on a cup of coffee or pay a 50-100% premium for a dubious "organic" label (a term that has drifted very, very far from what it meant when Rodale coined it).
That makes more money available for farm labor, to improve safety and pay workers better. It would take only a small increase in costs to make a large increase in wages: they receive only a few percent of the final consumer cost of the product.