For the most part, ML doesn't control high-risk things with a human in the loop. The worst cases for ML so far have been stock market crashes from algo trading.

Its entirely possible that an update to self-driving car's algorithms cause a day of chaos as the self-driving cars lose control and crash. Worst case scenario.

I agree that the secondary effects of the ML systems are going to be far worse than the primary. We can only see how it goes.