I guess the gamble was that there would be a certain point where the edge cases disappeared into the noise and it didn't matter that the approximation was/is "wrong" because the behavior of the car would match the requirements of the situation even if it wasn't for the right reasons.
To be fair I remember reading about GPT2 and thinking that LLMs would blow out for similar reasons.