This is not inherently a property of either LLMs or "classical" ML.
I can run a linear regression and get the same result every time or I can go to chat.com and get A/B tested upon. yes these are cherry picked examples. they're also the ones that apply to me personally. for a higher level version of this argument please see my response to the other responder.
linear regression on commodity hardware is generally not going to be truly deterministic delta implementation details
Transformers are plenty deterministic. We just decide to run an rng against their output to get the final token in most cases.
when concept A applies to concept B in practice, a lack of theoretical connection between A and B is basically irrelevant to me as a lay person. people don't seem to grasp this argument well here so I'll be very clear, yes you are entirely correct, concurrently there isn't much reason for me to care about correctness when my eyes and ears tell me something else
Non-associativeness of floating point arithmetic plays a major role as well.