This highlights one of the hazards of a highly-automated future. A reason to put judges, purses, and other humans into the mix is to look for injustices and give people the right to make ad hoc exceptions.

If a mechanical process were to, say, photograph you at a red light camera, ticket you, and perhaps automatically take the funds from your account you might have no recourse. And once we have such a can imagine worse with opaque ML models and few to no humans in the loop.

Of course the system we have is also ripe for abuse, and can be unthinkingly rigid as well (think of “Brazil”). But better than full automation.