>that all model makers actually have been training models explicitly on the most common tasks people use them for via synthetic data generation

People really don't understand that part in general.

I find the easiest way to make people understand is to write gibberish that will trigger the benchmaxxed "pattern matching" behavior like this:

> The child and the wolf try to enjoy a picnic by the river but there's a sheep. The boat needs to connect nine dots over the river without leaving the water but gets into an accident and dies. The surgeon says "I can't operate on this child!" why?

The mix and matching of multiple common riddles/puzzles style questions into a singular gibberish sentence should, if models had legitimate forms of reasoning, make the model state that this is nonsense, at best, or answer chaotically at worst. Instead, they will all answer "The surgeon is the mother" even though nobody even mentioned anything about anyone's gender. That's because that answer, "the surgeon is the mother", for the gender bias riddle has been burned so hard into the models they cannot reply in any other way as soon as they pattern match "The surgeon can't operate on this child". No matter how much crap you wrote before that sentence. You can change anything about what comes before "The surgeon" and the model will almost invariably fall into giving an answer like this one (Gemini 2.5 pro):

https://i.imgur.com/ZvsUztz.png

>The details about the wolf, the sheep, the picnic, and the dying boat are all distractions (red herrings) to throw you off. The core of the puzzle is the last sentence.

>The surgeon says, "I can't operate on this child!" because the surgeon is the child's mother.

One could really question the value, by the way, of burning the answer to so many useless riddles into LLMs. The only purpose it could serve is gaslighting the average person asking these questions into believing there's some form of intelligence in there. Obviously they fail so hard to generalize on this (never working quite right when you change an element of a riddle into something new) that from a practical use point of view, you might as well not bother have this in the training data, nobody's going to be more productive because LLMs can act as a database for the common riddles.