Using secondary school maths to demystify AI
raspberrypi.org
raspberrypi.org
The point of the Turing Test is that if there is no extrinsic difference between a human and a machine the intrinsic difference is moot for practical purposes. That is not an argument to whether a machine (with linear algebra, machine learning, large language models, or any other method) can think or what constitutes thinking or consciousness.
The Chinese Room thought experiment is a compliment on the intrinsic side of the comparison: https://en.wikipedia.org/wiki/Chinese_room
>Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words.
which is can you tell the AI answers from the humans ones in a test. It then becomes an experimental result rather than what you mean by 'think' or maybe by 'extrinsic difference'.
Good luck.
Has everyone hastily agreed that it has been passed? Do people argue that a human can't figure out it's talking to an LLM if the user is aware that LLMs exist in the world and is aware of their limitations and that the chat log is able to extend to infinity ( "infinity" is a proxy here for any sufficient time, it could be minutes, days, months, or years)?
In fact, it is blindly easy for these systems to fail the Turing test at the moment. No human would have the patience to continue a conversation indefinitely without telling the person on the other side to kindly fuck off.
>It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart front the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either "X is A and Y is B" or "X is B and Y is A." The interrogator is allowed to put questions to A and B.
>We now ask the question, "What will happen when a machine takes the part of A in this game?" Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman?
(some bits removed)
It was done more as thought experiment. As a practical test it would probably be too easy to fake with ELIZA type programs to be a good test. So computers could probably pass but it's not really hard enough for most people's idea of AI.
Hence since current LLMs are bound to hallucinate given enough time and seem not to able to maintain a conversation context window as robustly as humans, they would inevitably fail?
I'm pretty sure a set of workshops isn't ACTUALLY going to solve a problem that philosophers have been at each other's throats for for the past half century.
But BOY does it get people talking!
Both sides of the debate have capital-O Opinions, and how else did you want to drum up interest for a set of mathematics workshops. O:-)
Hopefully we can talk about the actual math and stuff (although the article doesn't go into much of that).
(IMHO its not provocative but well catching a point.. about so called "intelligence" - what if we could look for intelligent knowledge - made in not statistic but semantic(?) and converging way - instead - of being distracted, afraid and.. outpriced, peanuts for the essence it doesn't have ?)
(in this case, thinkiness)
> finite context windows
like a human has
> or the fact that the model is "frozen" and stateless,
much like a human adult. Models get updated at a slower frequency than humans. AI systems have access to fetch new information and store it for context.
> or the idea that you can transfer conversations between models are trivial
because computers are better-organized than humanity.
I do hope you're able to remember what you had for lunch without incessantly repeating it to keep it in your context window
One of my earliest memories is of painting a ceramic mug when I was about 3 years old. The only reason I remember it is because every now and then I think about what my earliest memory is, and then I refresh my memory of it. I used to remember a few other things from when I was slightly older, but no longer do, because I haven't had reasons to think of them.
I don't think humans have specific black and white differences between types of knowledge that way LLMs do, but there is definitely a lot of behavior that is similar to context window vs training data (and a gradient in between). We remember recent things a lot better than less recent things. The quantity of stuff we can remember in our "working memory" is approximately finite. If you try to hold a complex thought in your mind, you can probably do that indefinitely, but if you then try to hold a second equally complex thought as well, you'll often lose the details of the first thought and need to reread or rederive those details.
How would you say human short term memory works if not by repeated firing (similar to repeatedly putting same tokens in over and over)?
I can restart a conversation with an LLM 15 days later and the state is exactly as it was.
Can't do that with a human.
The idea that humans have a longer, more stable context window than LLM's, CAN or is even LIKELY to be true given certain activities but please let's be honest about this.
If you talk to someone for an hour about a technical conversation I would guesstimate that 90% of humans would immediately start to lose track of details in about 10 minutes. So they write things down, or they mentally repeat things to themselves they know or have recognized they keep forgetting.
I know this because it's happened continually in tech companies decade after decade.
LLM's have already passed the Turing test. They continue to pass it. They fool and outsmart people day after day.
I'm no fan of the hype AI is receiving, especially around overstating its impact in technical domains, but pretending that LLM's can't or don't consistently perform better than most human adults on a variety of different activities is complete non-sense.
I do hope you're able to remember what was your browser tab 5 tab switches ago without keeping track of it...
it doesn't sound like you really understand what these statements mean. if LLMs are like any humans it's those with late stage dementia, not healthy adults
Overwhelmingly, I just don't think the majority of human beings have the mental toolset to work with ambiguous philosophical contexts. They'll still try though, and what you get out of that is a 4th order baudrillardian simulation of reason.
Sentences constructed of words and representations of ideas defined long before you existed. I question whether you can work with ambiguous contexts as you have had the privilege of them being laid out in language for you already by the time you were born.
From my reference frame you appear to merely be circumlocuting from memory, and become the argument you make about others.
There's many definitions of "thinking".
AI and brains can do some, AI and brains definitely provably cannot do others, some others are untestable at present, and nobody really knows enough about what human brains do to be able to tell if or when some existing or future AI can do whatever is needed for the stuff we find special about ourselves.
A lot of people use different definitions, and respond to anyone pointing this out by denying the issue and claiming their own definition is the only sensible one and "obviously" everyone else (who isn't a weird pedant) uses it.
The definition of "thinking" in any of the parent comments or TFA is actually not defined. Like literally no statements are made about what is being tested.
So, if we had that we could actually discuss it. Otherwise it's just opinions about what a person believes thinking is, combined with what LLMs are doing + what the person believes they themselves do + what they believe others do. It's entirely subjective with very low SNR b/c of those confounding factors.
There are people who insist that the halting problem "proves" that machines will never be able to think. That this means they don't understand the difference between writing down (or generating a proof of) the halting problem and the implications of the halting problem, does not stop them from using it.
Is it only humans that have this need? That makes the need special, so humans are special in the universe.
We don't fully understand how brains work, but we know brains don't function like a computer. Why would a computer be assumed to function like a brain in any way, even in part, without evidence and just hopes based on marketing? And I don't just mean consumer marketing, but marketing within academia as well. For example, names like "neural networks" have always been considered metaphorical at best.
And then what do you even mean by "a computer?" This falls into the same trap because it sounds like your statement that brains don't function like a computer is really saying "brains don't function like the computers I am familiar with." But this would be like saying quantum computers aren't computers because they don't work like classical computers.
To put this in terms of "results", because that's what your way of thinking insists upon, a plane does not take off and land the way a bird does. This limits a plane's practicality to such an extent that a plane is useless for transportation without all the infrastructure you're probably ignoring with your argument. You might also be ignoring all the side effects planes bring with them.
Would you not agree that if we only ever wanted "flight" for a specific use case that apparently only birds can do after evaluating what a plane cannot do, then planes are not capable of "flight"?
This is the very same problem with "thought" in terms of AI. We're finding it's inadequate for what we want the machine to do. Not only is it inadequate for our current use cases, and not only is it inadequate now, but it will continue to be inadequate until we further pin down what "thought" is and determine what lies beyond the Church-Turing thesis.
https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#P...
Relevant quote: "B. Jack Copeland states that it is an open empirical question whether there are actual deterministic physical processes that, in the long run, elude simulation by a Turing machine; furthermore, he states that it is an open empirical question whether any such processes are involved in the working of the human brain"
Unicorns are not bound by the laws of physics - because they do not exist.
If thinking is definable, it is wrong that all statements about it are unverifiable (i.e. there are statements about it that are verifiable.)
Well, basic shit.
edit : Thinking is undefined, statements about undefined cannot be verified.
The "hair-splitting" underlies the whole GenAI debate.
It ties into another aspect of these perennial threads, where it is somehow OK for humans to engage in deluded or hallucinatory thought, but when an AI model does it, it proves they don't "think."
They're not equivalent at all because the AI is by no means biological. "It's just maths" could maybe be applied to humans but this is backed entirely by supposition and would ultimately just be an assumption of its own conclusion - that human brains work on the same underlying principles as AI because it is assumed that they're based on the same underlying principles as AI.
It's on those who want alternative explanations to demonstrate even the slightest need for them exists - there is no scientific evidence that exists which suggests the operation of brains as computers, as information processors, as substrate independent equivalents to Turing machines, are insufficient to any of the cognitive phenomena known across the entire domain of human knowledge.
We are brains in bone vats, connected to a wonderful and sophisticated sensorimotor platform, and our brains create the reality we experience by processing sensor data and constructing a simulation which we perceive as subjective experience.
The explanation we have is sufficient to the phenomenon. There's no need or benefit for searching for unnecessarily complicated alternative interpretations.
If you aren't satisfied with the explanation, it doesn't really matter - to quote one of Neil DeGrasse Tyson's best turns of phrase: "the universe is under no obligation to make sense to you"
If you can find evidence, any evidence whatsoever, and that evidence withstands scientific scrutiny, and it demands more than the explanation we currently have, then by all means, chase it down and find out more about how cognition works and expand our understanding of the universe. It simply doesn't look like we need anything more, in principle, to fully explain the nature of biological intelligence, and consciousness, and how brains work.
Mind as interdimensional radios, mystical souls and spirits, quantum tubules, none of that stuff has any basis in a ruthlessly rational and scientific review of the science of cognition.
That doesn't preclude souls and supernatural appearing phenomena or all manner of "other" things happening. There's simply no need to tie it in with cognition - neurotransmitters, biological networks, electrical activity, that's all you need.
Right back at you, brochacho. I'm not the one making a positive claim here. You're the one who insists that it must work in a specific way because you can't conceive of any alternatives. I have never seen ANY evidence or study linking any existent AI or computer system to human cognition.
>There's no need or benefit for searching for unnecessarily complicated alternative interpretations.
Thanks, if it's alright with you I might borrow this argument next time somebody tries to tell me the world isn't flat.
>It simply doesn't look
That's one of those phrases you use when you're REALLY confident that you know what you're talking about.
> like we need anything more, in principle, to fully explain the nature of biological intelligence, and consciousness, and how brains work.
Please fully explain the nature of intelligence, consciousness, and how brains work.
>Mind as interdimensional radios, mystical souls and spirits, quantum tubules, none of that stuff has any basis in a ruthlessly rational and scientific review of the science of cognition.
well i definitely never said anything even remotely similar to that. If i didn't know any better i might call this argument a "hallucination".
This is the point, we don't know the delta between brains and AI any assumption is equivalent to my statement.
But I think most people get what GP means.
When you think in these terms, it becomes clear that LLMs can’t have certain types of experiences (eg see in color) but could have others.
A “weak” panpsychism approach would just stop at ruling out experience or qualia based on physical limitations. Yet I prefer the “strong” pansychist theory that whatever is not forbidden is required, which begins to get really interesting (would imply that for example an LLM actually experiences the interaction you have with it, in some way).
As for applying the word thinking to AI systems, it's already in common usage and this won't change. We don't have any other candidate words, and this one is the closest existing word for referencing a computational process which, one must admit, is in many ways (but definitely not in all ways) analogous to human thought.
I think there is abundant evidence that the answer is ‘no’. The main reason is that consciousness doesn’t give you new physics, it follows the same rules and restrictions. It seems to be “part of” the standard natural universe, not something distinct.
if there's surely no algo to solve the halting problem, why would there be maths that describes consciousness?
Having read “I Am a Strange Loop” I do not believe Hofstadter indicates that the existence of Gödel’s theorem precludes consciousness being realizable on a Turing machine. Rather if I recall correctly he points out that as a possible argument and then attempts to refute it.
On the other hand Penrose is a prominent believer that human’s ability to understand Gödel’s theorem indicates consciousness can’t be realized on a Turing machine but there’s far from universal agreement on that point.
I'll try and ask OG q more clearly: why would the brain, consciousness, be formalizable?
I think there's a yearn view nature as adhering to an underlying model, and a contrary view that consciousness is transcendental, and I lean towards the latter
That wasn't the assumption though, it was only that human brains work by some "non-magical" electro-chemical process which could be described as a mechanism, whether that mechanism followed the same principles of AI or not.
These theories are enormously successful, but they are also known to be variously incomplete, inconsistent, non-deterministic, philosophically problematic, open to multiple interpretations and only partially understood in their implications, with links between descriptions of things at different scales a particularly challenging and little understood topic. The more you learn about physics (and while I'm no physicist, I have a degree in the subject and have learned a great deal more since) the more you understand the limits of what we know.
Anybody who thinks there's no mystery to physics just doesn't know much about it. Anybody who confidently asserts as fact things like "the brain consists of protons, neutrons and electrons so it's impossible for it to do anything a computer can't do" is deducing things from their own ignorance.
You'd think it would unlock certain concepts for this class of people, but ironically, they seem unable to digest the information and update their context.
We observe through our senses geometric relationships.
Syntax is exactly that; letters, sentences, paragraphs organized in spatial/geometric relationships.
At best thought is recreation of neural networks in the brain which only exist as spatial relationships.
Our senses operate on spatial relationships; enough light to work by, and food relative to stomach to satisfy our biological impetus to survive (which is spatial relationships of biochemistry).
The idea of "thought" as anything but biology makes little sense to me then as a root source is clearly observable. Humanity, roughly, repeats the same social story. All that thought does not seem to be all that useful as we end up in the same place; the majority as serfs of aristocracy.
Personally would prefer less "thought" role-play and more people taking the load of the labor they exploit to enable them to sit and "think".
But the accompanying XY plot showed samples that overlapped or at least were ambiguous. I immediately lost a lot of my interest in their approach, because traffic lights by design are very clearly red, or green. There aren't mauve or taupe lights that the local populace laughs at and says, "yes, that's mostly red."
I like the idea of studying math by using ML examples. I'm guessing this is a first step and future education will have better examples to learn from.
I suspect you feel this because you are observing the output of a very sophisticated image processing pipeline in your own head. When you are dealing with raw matrixes of rgb values it all becomes a lot more fuzzy. Especially when you encounter different illuminations, exposures and the cropping of the traffic light has noise on it. Not saying it is some intractably hard machine vision problem, because it is not. But there is some variety and fuzzyness there in the raw sensor measurements.
But they are two different things with overlapping qualities.
It's like MDMA and falling in love. They have many overlapping quantities but no one would claim one is the other.
The same arguments that appeared in 2015 inevitably get trotted out, almost verbatim, ten years later. It would be amusing on other sites, but it's just pathetic here.
Personally, I'm ok with reusing the word "thinking", but there are dogmatic stances on both sides. For example, lots of people decreeing that biology in the end can't but reduce to maths, since "what else could it be". The truth is we don't actually know if it is possible, for any conceivable computational system, to emulate all essential aspects of human thought. There are good arguments for this (in)possibility, like those presented by Roger Penrose in "the Emperor's new Mind" and "Shadows of the Mind".
For one thing, yes, they can, obviously [1] -- when's the last time you checked? -- and for another, there are plenty of humans who seemingly cannot.
The only real difference is that with an LLM, when the context is lost, so is the learning. That will obviously need to be addressed at some point.
that they can't perform simple mathematical operations without access to external help (via tool calling)
But yet you are fine with humans requiring a calculator to perform similar tasks? Many humans are worse at basic arithmetic than an unaided transformer network. And, tellingly, we make the same kinds of errors.
or that they have to expend so much more energy to do their magic (and yes, to me they are a bit magical), which makes some wonder if what these models do is a form of refined brute-force search, rather than ideating.
Well, of course, all they are doing is searching and curve-fitting. To me, the magical thing is that they have shown us, more or less undeniably (Penrose notwithstanding), that that is all we do. Questions that have been asked for thousands of years have now been answered: there's nothing special about the human brain, except for the ability to form, consolidate, consult, and revise long-term memories.
1: E.g., https://arxiv.org/abs/2005.14165 from 2020
That's post-training. The complaint I'm referring to is to the huge amounts of data (end energy) required during training - which is also a form of learning, after all. Sure, there are counter-arguments, for example pointing to the huge amount of non-textual data a child ingests, but these counter-arguments are not waterproof themselves (for example, one can point out that we are discussing text-only tasks). The discussion can go on and on, my point was only that cogent arguments are indeed often presented, which you were denying above.
> there are plenty of humans who seemingly cannot
This particular defense of LLMs has always puzzled me. By this measure, simply because there are sufficiently impaired humans, AGI has already been achieved many decades ago.
> But yet you are fine with humans requiring a calculator to perform similar tasks
I'm talking about tasks like multiplying two 4-digit numbers (let's say 8-digit, just to be safe, for reasoning models), which 5th or 6th graders in the US are expected to be able to do with no problem - without using a calculator.
> To me, the magical thing is that they have shown us, more or less undeniably (Penrose notwithstanding), that that is all we do.
Or, to put it more tersely, they have shown you that that is all we do. Penrose, myself, and lots of others don't see it quite like that. (Feeling quite comfortable being classed in the same camp with the greatest living physicist, honestly. ;) To me what LLMs do is approximate one aspect of our minds. But I have a strong hunch that the rabbit hole goes much deeper, your assessment notwithstanding.
No, it is not. Read the paper. They are discussing an emergent property of the context itself: "For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model."
I'm talking about tasks like multiplying two 4-digit numbers (let's say 8-digit, just to be safe, for reasoning models), which 5th or 6th graders in the US are expected to be able to do with no problem - without using a calculator.
So am I. See, for example, Karpathy's discussion of native computation: https://youtu.be/7xTGNNLPyMI?si=Gckcmp2Sby4SlKje&t=6416 (starts at 1:46:56). The first few tokens in the context actually serve as some sort of substrate for general computation. I don't pretend to understand that, and it may still be something of an open research topic, but it's one more unexpected emergent property of transformers.
You'd be crazy to trust that property at this stage -- at the time Karpathy was making the video, he needed to explicitly tell the model to "Use code" if he didn't want it to just make up solutions to more complex problems -- but you'd also be crazy to trust answers from a 5th-grader who just learned long division last week.
Feeling quite comfortable being classed in the same camp with the greatest living physicist, honestly.
Not a great time for you to rest on your intellectual laurels. Same goes for Penrose.
Yes, it is. You seem to have misunderstood what I wrote. The critique I was pointing to is of the amount of examples and energy needed during model training, which is what the "learning" in "machine learning" actually refers to. The paper uses GPT-3 which had already absorbed all that data and electricity. And the "learning" the paper talks about is arguably not real learning, since none of the acquired skills persists beyond the end of the session.
> So am I.
This is easy to settle. Go check any frontier model and see how far they get with multiplying numbers with tool calling disabled.
> Not a great time for you to rest on your intellectual laurels. Same goes for Penrose.
Neither am I resting, nor are there much laurels to rest on, at least compared to someone like Penrose. As for him, give the man a break, he's 94 years old and still sharp as a tack and intellectually productive. You're the one who's resting, imagining you've settled a question which is very much still open. Certainty is certainly intoxicating, so I understand where you're coming from, but claiming anyone who doubts computationalism is not bringing any arguments to the table is patently absurd.
Nobody is arguing about power consumption in this thread (but see below), and in any case the majority of power consumption is split between one-time training and the burden of running millions of prompts at once. Processing individual prompts costs almost nothing.
And it's already been stipulated that lack of long-term memory is a key difference between AI and human cognition. Give them some time, sheesh. This stuff's brand new.
This is easy to settle. Go check any frontier model and see how far they get with multiplying numbers with tool calling disabled.
Yes, it is very easy to settle. I ran this session locally in Qwen3-Next-80B-A3B-Instruct-Q6_K: https://pastebin.com/G7Ewt5Tu
This is a 6-bit quantized version of a free model that is very far from frontier level. It traces its lineage through DeepSeek, which was likely RL-trained by GPT 4.something. So 2 out of 4 isn't bad at all, really. My GPU's power consumption went up by about 40 watts while running these queries, a bit more than a human brain.
If I ask the hardest of those questions on Gemini 3, it gets the right answer but definitely struggles: https://pastebin.com/MuVy9cNw
As for him, give the man a break, he's 94 years old and still sharp as a tack and intellectually productive.
(Shrug) As long as he chooses to contribute his views to public discourse, he's fair game for criticism. You don't have to invoke quantum woo to multiply numbers without specialized tools, as the tests above show. Consequently, I believe that a heavy burden of proof lies with anyone who invokes quantum woo to explain any other mental operations. It's a textbook violation of Occam's Razor.
Conversely, if the one asserting something doesn't want to define it there is no useful conversation to be had. (as in: AI doesn't think - I won't tell you what I mean by think)
PS: Asking someone to falsify their own assertion doesn't seem a good strategy here.
PPS: Even if everything about the human brain can be emulated, that does not constitute progress for your argument, since now you'd have to assert that AI emulates the human brain perfectly before it is complete. There is no direct connection between "This AI does not think" to "The human brain can be fully emulated". Also the difference between "does not" and "can not" is big enough here that mangling them together is inappropriate.
A lot of people seemingly haven't updated their priors after some of the more interesting results published lately, such as the performance of Google's and OpenAI's models at the 2025 Math Olympiad. Would you say that includes yourself?
If so, what do the models still have to do in order to establish that they are capable of all major forms of reasoning, and under what conditions will you accept such proof?
For that matter I have no opinion on if AI does think or not, I simply don't care. Therefore I also really can't answer your question in what more a model has to do to establish that they are thinking (does being able to use all major forms of reasoning constitute the capability of thought to you?). I can say however, that any such proof would have to be on a case-by-case basis given my current understanding on AI is designed.
Sometimes, because of the consequences of otherwise, the order gets reversed
Whatever you meant to say with "Sometimes, because of the consequences of otherwise, the order gets reversed" eludes me as well.
So we don't require, say, minorities or animals to prove they have souls, we just inherently assume they do and make laws around protecting them.
With regards to the topic: Does AI think? I don't know, but I also don't want to act upon knowing if it does (or doesn't for that matter). In other words, I don't care. The answer could go either way, but I'd rather say that I don't know (especially since "thinking" is not defined). That means that I can assume both and consider the consequences using some heuristic to decide which assumption is better given the action I want to justify doing or not doing. If you want me to believe an AI thinks, you have to prove it, if you want to justify an action you may assume whatever you deem most likely. And if you want to know if an AI thinks, then you literally can't assume it does; simple as that.
There are people confidently claiming they can’t and then other people expressing skepticism at their confidence and/or trying to get them to nail down what they mean.
... someone else points out that the same models that can't "think" are somehow turning in gold-level performance at international math and programming competitions, making Fields Medalists sit up and take notice, winning art competitions, composing music indistinguishable from human output, and making entire subreddits fail the Turing test.
Uh huh. Good luck getting Stockfish to do your math homework while Leela works on your next waifu.
LLMs play chess poorly. Chess engines do nothing else at all. That's kind of a big difference, wouldn't you say?
To their utility.
Not sure if it matters on the question "thinking?"; even if for the debaters "thinking" requires consciousness/qualia (and that varies), there's nothing more than guesses as to where that emerges from.
For my original earlier reply, the main subtext would be: "Your complaint is ridiculously biased."
For the later reply about chess, perhaps: "You're asserting that tricking, amazing, or beating a human is a reliable sign of human-like intelligence. We already know that is untrue from decades of past experience."
I don't know who's asserting that (other than Alan Turing, I guess); certainly not me. Humans are, if anything, easier to fool than our current crude AI models are. Heck, ELIZA was enough to fool non-specialist humans.
In any case, nobody was "tricked" at the IMO. What happened there required legitimate reasoning abilities. The burden of proof falls decisively on those who assert otherwise.
This is exactly the problem. Claims about AI are unfalsifiable, thus your various non-sequiturs about AI 'thinking'.
The reason I say this is because an LLM is not a complete self-contained thing if you want to compare it to a human being. It is a building block. Your brain thinks. Your prefrontal cortex however is not a complete system and if you somehow managed to extract it and wire it up to a serial terminal I suspect you’d be pretty disappointed in what it would be capable of on its own.
I want to be clear that I am not making an argument that once we hook up sensory inputs and motion outputs as well as motivations, fears, anxieties, desires, pain and pleasure centers, memory systems, sense of time, balance, fatigue, etc. to an LLM that we would get a thinking feeling conscious being. I suspect it would take something more sophisticated than an LLM. But my point is that even if an LLM was that building block, I don’t think the question of whether it is capable of thought is the right question.
The AI companies themselves are the ones drawing the parallels to a human being. Look at how any of these LLM products are marketed and described.
"Intelligence" implies "thinking" for most people, just as "Learning" in machine learning implies "understanding" for most people. The algorithms created neither 'think' nor 'understand' and until you understand that, it may be difficult to accurately judge the value of the results produced by these systems.
Why, when we use the term for AI, do we skip over this distinction and expect it to be as good as the original—- or better?
That wouldn’t be artificial intelligence, it would just be the original artifact: “intelligence”.
It's not a perfect term, but we have been using it for seven full decades to include all of machine learning and plenty of things even less intelligent
1: https://www-formal.stanford.edu/jmc/history/dartmouth/dartmo...
Military justice is to justice as military music is to music
"how much thinking time did the LLMs get when getting gold in the maths olympiad" is not a nonsense question. Four and a half hours apparently. Different thing.
You could go on to ask if saying humans thinking about the problem is thinking but LLMs thinking about the problem is not thinking and if so why? Maybe only synapses count?
There's absolutely no similarity between what computer hardware does and what a brain does. People will twist and stretch things and tickle the imagination of the naive layperson and that's just wrong. We seriously have to cut this out already.
Anthropomorphizing is dangerous even for other topics, and long understood to be before computers came around. Why do we allow this?
The way we talk about computer science today sounds about as ridiculous as invoking magic or deities to explain what we now consider high school physics or chemistry. I am aware that the future usually sees the past as primitive, but why can't we try to seem less dumb at least this time around?
But at very least there's also no similarity between what computer hardware does and what even the simplest of LLMs do. They don't run on eg. x86_64 , else qemu would be sufficient for inferencing.
> Large language models (LLMs) can be dishonest when reporting on their actions and beliefs -- for example, they may overstate their confidence in factual claims or cover up evidence of covert actions
Still, the sales pitch has worked to unlock huge liquidity for him so there’s that.
Still making predictions is a big part of what brains do though not the only thing. Someone wise said that LLM intelligence is a new kind of intelligence, like how animal intelligence is different from ours but is still intelligence but needs to be characterized to understand differences.
So long as you accept the slide ruler as a "new kind of intelligence" everything will probably work out fine, it's the Altmannian insistence that only the LLM is of the new kind that is silly.
I saw a YouTube video about a investigative youtuber Eddy Burback who very easily convinced chat4 that he should cut off all contact with friends and family, move to a cabin in the desert, eat baby food, wrap himself in alfoil, etc just feeding his own (faked) mistakes and delusions. "What you are doing is important, trust your instincts".
Wven if AI could hypothetically be 100x as smart as a human under the hood, it still doesn't care. It doesn't do what it thinks it should, it doesn't do what it needs to do, it does what we train it to.
We train in humanities weaknesses and follies. AI can hypothetically exceed humanity in some respects, but in other respects it is a very hard to control power tool.
AI is optimised, and optimised functions always "hack" the evaluation function. In the case of AI, the evaluation function includes human flaws. AI is trained to tell us what we want to hear.
Elon Musk sees the problem, but his solution is to try to make it think more like him, and even if that succeeds it just magnifies his own weaknesses.
Has anyone read the book criticising Ray Dalio? He is a very successful hedge fund manager, who decided that he could solve the problem of finding a replacement by psychology evaluation and training people to think like him. But even his smartest employees didn't think like him, they just (reading between the lines) gamed his system. Their incentives weren't his incentives - he could demand radical honesty and integrity but that doesn't work so well when he would (of course) reward the people who agreed with him, rather than the people who would tell him he was screwing up. His organisation (apparently) became a bunch of even more radical syncopants due to his efforts to weed out syncophantcy.
The opinions are exactly the same than about LLM.
Penrose comes to mind, he will die on the hill that the brain involves quantum computations somehow, to explain his dualist position of "the soul being the entity responsible for deciding how the quantum states within the brain collapse, hence somehow controlling the body" (I am grossly simplifying). But even if that was the case, if the brain did involve quantum computations, those are still, well, computable. They just involve some amount of randomness, but so what? To continue with grandparent's experiment, you'd have to replace biological neurons with tiny quantum computer neurons instead, but the gist is the same.
Would you mind expanding on this? At a base read, it seems you implying magic exists.
AFAICT, your comment above would need some mechanism that is physically impossible and incalculable to make the argument, and then somehow have that happen in a human brain despite being physically impossible and incalculable.
"Can not be measured", probably not. "We don't know how to measure", almost certainly.
I am capable of belief, and I've seen no evidence that the computer is. It's also possible that I'm the only person that is conscious. It's even possible that you are!
The argument that was actually made was "LLMs do not think".
B: But Y would also imply Z
C: A was never arguing for Z! This is a strawman!
Everything I've seen says "LLMs cannot think like brains" is not dependent on an argument that "no computer can think like a brain", but rather on an understanding of just what LLMs are—and what they are not.
The concept of understanding emerges on a higher level from the way the neurons (biological or virtual) are connected, or the way the instructions being followed by the human in the Chinese room process the information
But really this is a philosophical/definitional thing about what you call “thinking”
Edit: I see my take on this is listed on the page as the “System reply”
Check out eg Dennett.... or ... his opionions about Searle; Have fun with eg... this:
"By Searle’s own count, there are over a hundred published attacks on it. He can count them, but I guess he can’t read them, for in all those years he has never to my knowledge responded in detail to the dozens of devastating criticisms they contain;"
https://www.nybooks.com/articles/1995/12/21/the-mystery-of-c...
If you were a mind supervening on the behavior of some massive time/space scale computer, how would you know? How could you tell the difference between running on a human making marks with pen and paper and running on a modern CPU? Your experience updates based on information transformations, not based on how fast the fundamental substrate is changing. When your conscious experience changes, that means your current state is substantially different from your prior state and you can recognize this difference. Our human-scale chauvinism gets in the way of properly imagining this. A mind running on a CPU or a large collection of human computers is equally plausible.
A common question people like to ask is "where is the consciousness" in such a system. This is an important question if only because it highlights the futility of such questions. Where is Microsoft Word when it is running on my computer? How can you draw a boundary around a computation when there are a multitude of essential and non-essential parts of the system that work together to construct the relevant causal dynamic. It's just not a well-defined question. There is no one place where Microsoft Word occurs nor is there any one place where consciousness occurs in a system. Is state being properly recorded and correctly leveraged to compute the next state? The consciousness is in this process.
Another issue is that the brain is overly complex for consciousness to just be received from elsewhere. Typically a radio is much less complex than the signal being received, or at least less complex than the potential space of signals it is possible to receive. We don't see that with consciousness. In fact, consciousness seems to be far less complex than the brain that supports it. The issue of the specificity of brain damage and the corresponding specificity in conscious deficits also points away from the receiver idea.
You can replicate the entire universe with pen and paper (or a bunch of rocks). It would take an unimaginably long time, and we haven't discovered all the calculations you'd need to do yet, but presumably they exist and this could be done.
Does that actually make a universe? I don't know!
The comic is meant to be a joke, I think, but I find myself thinking about it all the time!!!
The question is, are the people in the simulated universe real people? Do they think and feel like we do—are they conscious? Either answer seems like it can’t possibly be right!
Connect your pen and paper operator to a brainless human body, and you got something indistinguishable from a regular alive human.
[0] https://en.wikipedia.org/wiki/Functionalism_%28philosophy_of...
That's an assumption, though. A plausible assumption, but still an assumption.
We know you can execute an LLM on pen and paper, because people built them and they're understood well enough that we could list the calculations you'd need to do. We don't know enough about the human brain to create a similar list, so I don't think you can reasonably make a stronger statement than "you could probably simulate..." without getting ahead of yourself.
Yes, or what about leprechauns?
It's been kinda discussed to oblivion in the last century, interesting that it seems people don't realize the "existing literature" and repeat the same arguments (not saying anyone is wrong).
An arbitrarily-perfect simulation of a burning candle will never, ever melt wax.
An LLM is always a description. An LLM operating on a computer is identical to a description of it operating on paper (if much faster).
That simulated candle is perfectly melting wax in its own simulation. Duh, it won't melt any in ours, because our arbitrary notions of "real" wax are disconnected between the two simulatons.
If we don't think the candle in a simulated universe is a "real candle", why do we consider the intelligence in a simulated universe possibly "real intelligence"?
Being a functionalist ( https://en.wikipedia.org/wiki/Functionalism_(philosophy_of_m... ) myself, I don't know the answer on the top of my head.
I can smell a "real" candle, a "real" candle can burn my hand. The term real here is just picking out a conceptual schema where its objects can feature as relata of the same laws, like a causal compatibility class defined by a shared causal scope. But this isn't unique to the question of real vs simulated. There are causal scopes all over the place. Subatomic particles are a scope. I, as a particular collection of atoms, am not causally compatible with individual electrons and neutrons. Different conceptual levels have their own causal scopes and their own laws (derivative of more fundamental laws) that determine how these aggregates behave. Real (as distinct from simulated) just identifies causal scopes that are derivative of our privileged scope.
Consciousness is not like the candle because everyone's consciousness is its own unique causal scope. There are psychological laws that determine how we process and respond to information. But each of our minds are causally isolated from one another. We can only know of each other's consciousness by judging behavior. There's nothing privileged about a biological substrate when it comes to determining "real" consciousness.
I'm not against this conclusion ( https://en.wikipedia.org/wiki/Philosophical_zombie ) but it doesn't seem to be compatible with what most people believe in general.
Determining what is real by judging causal scope is generally successful but it misleads in the case of consciousness.
If I make a button that lights the candle, and another button that puts it off, and I press those buttons, then the virtual candle is causally connected to our physical reality world.
But obviously the candle is still considered virtual.
Maybe a candle is not as illustrative, but let's say we're talking about a very realistic and immersive MMORPG. We directly do stuff in the game, and with the right VR hardware it might even feel real, but we call it a virtual reality anyway. Why? And if there's an AI NPC, we say that the NPC's body is virtual -- but when we talk about the AI's intelligence (which at this point is the only AI we know about -- simulated intelligence in computers) why do we not automatically think of this intelligence as virtual in the same way as a virtual candle or a virtual NPC's body?
Real is about an object having all of the essential properties for that concept. If we take it as essential that candles can burn our hand, then the virtual candle isn't real. But it is not essential to consciousness that it is not virtual.
A candle in Canada can't melt wax in Mexico, and a real candle can't melt simulated wax. If you want to differentiate two things along one axis, you can't just point out differences that may or may not have any effect on that axis. You have to establish a causal link before the differences have any meaning. To my knowledge, intelligence/consciousness/experience doesn't have a causal link with anything.
We know our brains cause consciousness the way we knew in 1500 that being on a boat for too long causes scurvy. Maybe the boat and the ocean matter, or maybe they don't.
A simulation of a tree growing (say) is a lot more like the idea of love than it is... a real tree growing. Making the simulation more accurate changes that not a bit.
Thanks for stating your views clearly. I have some questions to try and understand them better:
Would you say you're sure that you aren't in a simulation while acknowledging that a simulated version of you would say the same?
What do you think happens to someone whose neurons get replaced by small computers one by one (if you're happy to assume for the sake of argument that such a thing is possible without changing the person's behavior)?
Build a simulation of creatures that evolve from simple structures (think RNA, DNA).
Now, if in this simulation, after many many iterations, the creatures start talking about consciousness, what does that tell us?
It might if the simulation includes humans observing the candle.
Whatever that something that it actually does in the real, physical world is produces the cogito in cogito, ergo sum and I doubt you can get it just by describing what all the subatomic particles are doing, any more than a computer or pen-and-paper simulated hurricane can knock your house down, no matter how perfectly simulated.
A pen and paper simulation of a brain would also be "a thing happening" as you put it. You have to explain what is the magical ingredient that makes the brain's computations impossible to replicate.
You could connect your brain simulation to an actual body, and you'd be unable to tell the difference with a regular human, unless you crack it open.
I'm not. You might want me to be, but I'm very, very much not.
Of course a GPU involves things happening. No amount of using it to describe a brain operating gets you an operating brain, though. It's not doing what a brain does. It's describing it.
(I think this is actually all somewhat tangential to whether LLMs "can think" or whatever, though—but the "well of course they might think because if we could perfectly describe an operating brain, that would also be thinking" line of argument often comes up, and I think it's about as wrong-headed as a thing can possibly be, a kind of deep "confusing the map for the territory" error; see also comments floating around this thread offhandedly claiming that the brain "is just physics"—like, what? That's the cart leading the horse! No! Dead wrong!)
Expecting machines to think is.. like magical thinking (but they are good at calculations indeed).
I wish we didn't use the word intelligence in context of LLMs - shortly there is Essence and the rest.. is only slope - into all possible combinations of Markov Chains - may they have sense or not I don't see how part of some calculation could recognize it, or that to be possible from inside (of calculation, that doesn't even consider that).
Aside of artificial knowledge (out of senses, experience, context lengths.. - confabulating but not knowing that), I wish to see an intelligent knowledge - made in kind of semantic way - allowed to expand using not yet obvious (but existing - not random) connections. I wouldn't expect it to think (humans think, digitals calculate). But I would expect it to have a tendency to be coming closer (not further) in reflecting/modeling reality and expanding implications.
An LLM could be thinking in one of two ways. Either between adding each individual token, or collectively across multiple tokens. At the individual token level the physical mechanism doesn’t seem to fit the definition being essentially reflexive action, but across multiple tokens that’s a little more questionable especially as multiple approaches are used.
> across multiple tokens
- but how many ? how many of them happen in sole person life ? How many in some calculation ? Does it matter, if a calculation doesn't reflect it but stay all the same ? (conversation with.. a radio - would it have any sense ?)
The connotation is simply an internal process of indeterminate length rather than one of reflexive length. So they don’t apply it when a GPU is slinging out 120 FPS in a first person shooter.
But now, I see this: the truth is static and non-profit, but calculating something can be sold again and again, if you have a hammer (processing) everything looks like a nail, to sell well the word thinking had to be used instead of excuse for every time results being different (like the shadows) - then, we can have only things that let someone else keep making profits: JS, LLM, whatever.. (just not.. "XSLT" alike).
(yet, I need to study for your second sentence;)
(In math class I've been told that not some number but the formula is the expected result..
.. then people are now asking: "Just show me the prompt instead !" and.. done. ;) )
- ragebait them by saying AIs don’t think
- …
LLMs are BAD at evaluating earlier thinking errors, precisely because there's not copious examples of text where humans thinking through a problem, screwing up, going back, correcting their earlier statement, and continuing. (a good example catches these and corrects them)
> Secondary school maths showing that AI systems don’t think
And the article contains the quotes:
> the team wants to tackle a major and common misconception: that students think that ANN systems learn, recognise, see, and understand, when really it’s all just maths.
> The team is taking very complex ideas and reducing them to such an extent that we can use secondary classroom maths to show that AI is not magic and AI systems do not think.
This is not off topic
If you're asking for things you can't easily verify you're barking up the wrong tree.
Very much like this effect https://www.reddit.com/r/opticalillusions/comments/1cedtcp/s... . Shouldn't hide complexity under a truth value.
(Sneaking a bit of belief in here, to me "substrate independence" is a more extreme position than the idea that a system could be made which is intelligent but not conscious, hence I find it implausible.)
Trying to avoid this kind of thing is why the guidelines say things like:
"Eschew flamebait. Avoid generic tangents."
"Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead."
This is completely idiotic. Do these people actually believe that showing it can't be actual thought because it is described by math?
By every scientific measure we have the answer is no. It’s just electrical current taking the path of least resistance through connected neurons mixed with cell death.
The fact a human brain peaks at IQ around 200 is fascinating. Can the scale even go higher? It would seem no since nothing has achieved a higher score it must not exist.