Or did I miss?
Continuing, we largely fool ourselves when we play the shell game of hunting for scientific justification of "social facts". Ie: when we assume a social fact, then hunt for scientific evidence to ground it in then use the scientific evidence to justify the existence and validity of the social fact we assumed. This happens a LOT and it's epistemically invalid.
A LOT of folks dismissal of thinking machines reminds me of Feuerbach's The Essence of Christianity, specifically the section on anthropomorphism[1], but in an inverted form - the reason LLMs can't "think" is that it would dilute what we hold dear about ourselves: our ability to think. It's an ego protection mechanism, but no one's jumping to admit that.
https://www.gutenberg.org/files/47025/47025-h/47025-h.htm#pb...
I think I follow better. Thinking doesn't have a rigorous and testable definition, so it isn't scientific. We all have our own colloquial understand. Since it's not testable and the definition varies person to person, it's not useful when reasoning about AI or intelligence.
Also I wanted to say I love how you write!
To explain it by analogy, we had the concept of women and men for at least thousands of years. Socially construed by physical appearance and behavior. Then in 1905 Nettie Stevens discovered sex chromosomes. Her discovery "found" the existence of men in women in the epistemological universe of science. But then a curious thing happened, people began placing people in social categories based on sex chromosomes. It's an incestuous loop of logic that hoists itself into validity simply because folks forget how it occurred.
If we went searching for a scientifically rigorous definition of "thinking" we would forget we made the same fatal leap of logic.
I suppose I have trouble seeing it that way because I view words as pointers to meaning. The word itself holds no intrinsic value; it's the meaning humans attach to it and the context in which it's used that gives it significance.
When we give "thinking" a scientifically testable definition we can reason around it in a more academic way.
I've read psych papers that defined "thinking" as cognition, measured by action potentials. A participant was said to be "thinking more" when their EEG showed increased activity.
It's important to understand that "thinking" here was only defined within the context of the paper.
To use an analogy, many people say their computer is thinking when it's undergoing a heavy processor load. Strictly speaking this is not the case, their computer cannot think, but they aren't wrong or incorrect because in that context 'thinking' literally means 'processing'.
Concretely, what do you gain by giving a current-generation LLM more runtime? It's not trained/designed to do anything with it, more time = more tokens = more nonsense once past the "end" token/end of context. You could build an agent on top of the LLM that calls the LLM iteratively, but afaict this approach isn't strictly an improvement over the base LLM. Current architectures seem to be limited with how much improvement they can eke out of more runtime without a broader redesign or retraining.
Now with new architectures you might be able to do more with more runtime, but I'm not sold that allowing a current-gen LLM to execute code or do web-searches and re-feeding it the quetion + its output + new data is going to strictly return better results. Sometimes maybe yes, sometimes not.
So imo the current limitation is not the runtime allotted to LLMs but their fundamental (current-gen) design/training.
We're not really discussing whether LLMs are "thinking" in the sense of the "social category" in the abstract sense - that's just the one word that this discussion got hung up on.
Concretely, we're discussing whether giving LLMs a larger time-budget would yield better results - OP suggested that letting LLMs iteratively refine or enrich an answer might lead to better results. The term "thinking" was used as an analogy or quick crutch to convey an analogy, not 100% equivalence, between a hypothetic LLM output evaluation algo and people's supposed "fast" and "slow" modes of thinking (if they even exist, whatever). Even if people don't have these two modes of thought, as you said they might be post-fact reifications of sociological ideas, the algorithm/idea of iteratively reflecting on and self-refining LLM outputs might still apply to LLMs as LLMs are not actually thinking.
Imo current LLM architecture and/or training data facilitate (mostly?) the automatic non-iterative mode but OpenAI is trying to do more of the iterative, reflective mode with O1 afaict. Imo and currently, there's not much to be gained by iteratively feeding an a current-generation LLM it's own output to iteratively analyze and augment it. At least not in general. Even enriching the input data via RAG doesn't always lead to better results. Imo, there's more fundamental work necessary - and maybe OpenAI is doing that.