I recently did an experiment with LLMs where I asked them to rate the odds of a given outcome explaining an observation, and I found they are systematically biased much like humans [1].

Even if they can do great maths I think bayesian statistical fundamentals are not reliable at this point.

[1] https://think-twice.me/is-ai-biased-or-sexist-making-chatgpt...