That is not the only problem however, because if it were self driving would be solved by 2018.

The problem is central to using ML tricks for anything: when you use maximum likelihood estimates for training anything, you get no guarantees for unlikely situations. That’s why you get self driving cars dragging someone stuck beneath them or LLMs just straight up spewing fiction.