A very fast librarian with a select material of reference books and adept at speed reading could give the same answers as LLMs. A bit slower of course. Most of the time an encyclopedia would suffice and be more accurate than the hallucinating ghosts in the machine.
EDIT: The insane downvoters can go back to their AI girlfriends. The comment was meant for thinking people.
This is called the "Chinese Room" argument, postulating that a human equipped with a Borgesian library of reference books in a language he doesn't understand and a symbolic lookup table for that language can emulate a thinking human mind without actually thinking.
It was controversial for decades, and is now known not to hold up at all. (Or to hold up perfectly, if unlike Searle your goal is to show that human minds are nothing all that special.) To the extent that the room's occupant succeeds at a task that requires thinking, he is thinking... end of story. Simulated intelligence is now known, thanks to LLMs, to be indistinguishable from real intelligence.
Ask the operator of a Chinese room, who doesn't know Chinese or math, for a novel proof of an unsolved conjecture. The LLM can give you one, but your hypothetical reference librarian won't even know what to look up at first. By the time they learn, the core premise of the argument will no longer hold true.
Oooff, that's quite the claim. Why don't you try to back it up? Cite some research.
Last time I checked it was still very possible to distinguish "reasoning" LLMs from Real People.
The "last time you checked" was evidently 2023 or so. I suggest checking again.
Uh huh. So you have no evidence, and can't prove shit. That's what I figured.
LLMs are nothing but a shallow mirror of human thought, they don't hold up to any serious examination and the only people who think they do are typically narcissists who can't form deep connections with real people. The superficiality of LLMs is not apparent to these fans, because they have never managed to connect with the depth and inner world of another human.
Then, the next thing that goes through their mind -- the last thing -- is a chrome hood ornament shaped like a greyhound.
That’s your domain buddy, not mine.