In these cases, I wonder if it would cause a movement toward more interpretative models with more mathematical backing.

For example, if not being able to explain a model leaves an organization open to litigation, they may instead rely on statistical based learning methods even if they performed less well.

The losses made by software training may be the least problem for the institution.If case goes to court,they may be required to reveal the reasoning behind software's decision and that means going through the code,as all that marketing bs called AI is just a trading algorithm trained on some datasets. No company would want to reveal the code.