In your scenario there were still individuals accountable for the decisions and their outcomes.

How are you supposed to say why a machine learning model produces different outputs from the same input? It's just a black box.