there is a well studied scenario where humans are asked to hire people from four groups. These groups will be judged in their performance on a job and the humans rated on their hiring abilities. Unbeknownst to the human participants, all applicants are drawn from a single skill distribution, with groups assigned essentially randomly. Stastically, all groups have identical performance. Despite this, humans generalize over their early experiences, and develop biases towards specific groups.
While not identical, I relate this to the experience I have playing Fire emblem with random growths. A unit can get lucky and favored early despite being overall mediocre (hello Diamant from my first run through engage).
The researchers recreated this experiment with LLMs, and showed that the LLMs reproduce the human behavior of overgeneralizing early and failing to, as the paper says, sufficiently explore the space[0].
[1]: They instead exploit in the technical sense (https://en.wikipedia.org/wiki/Multi-armed_bandit), but exploit based on incomplete information.