I wonder if the rationalizations people come up with for why this isn't real intelligence will be as creative as ChatGPTs solution.
Some people think that multiplying numbers, remembering a large number of facts, and being good at calculations is intelligence.
Most intelligent people do not think that.
Eventually, we will arrive at the same conclusion for what LLMs are doing now.
Hah. It reminds me of this great quote, from the '80s:
> There is a related “Theorem” about progress in AI: once some mental function is programmed, people soon cease to consider it as an essential ingredient of “real thinking”. The ineluctable core of intelligence is always in that next thing which hasn’t yet been programmed. This “Theorem” was first proposed to me by Larry Tesler, so I call it Tesler’s Theorem: “AI is whatever hasn’t been done yet.”
We are seeing this right now in the comments. 50 years later, people are still doing this! Oh, this was solved, but it was trivial, of course this isn't real intelligence.
Are you also going to argue definitions of life before we even learned of microscopic or single cell organisms are correct and that the definitions we use today are wrong? That they are shifting goal posts? That “centuries later, people are still doing this”? No, that would be absurd.
For example, ~2 years ago, an expert in ML publicly made this remark on stage: LLMs can't do math. Today they absolutely and obviously, can. Yet somehow it's not impressive anymore. Or, and this is the key part of the quote, this is somehow not related to "intelligence". Something that 2 years ago was not possible (again, according to a leading expert in this field), is possible today. And yet this is somehow something that they always could do, and since they're doing it today, is suddenly no longer important. On to the next one!
No idea why this is related to darwin or definitions of life. The definitions don't change. What people considered important 2 years ago, is suddenly not important anymore. The only thing that changed is that today we can see that capability. Ergo, the quote holds.
See, that’s a poor argument already. Anyone could counter that with other experts in ML publicly making remarks that AI would have replaced 80% of the work force or cured multiple diseases by now, which obviously hasn’t happened. That’s about as good an argument as when people countered NFT critics by citing how Clifford Stoll said the internet was a fad.
> made this remark on stage: LLMs can't do math. Today they absolutely and obviously, can.
How exactly are “LLMs can’t” and “do math” defined? As you described it, that sentence does not mean “will never be able to”, so there’s no contradiction. Furthermore, it continues to be true that you cannot trust LLMs on their own for basic arithmetic. They may e.g. call an external tool to do it, but pattern matching on text isn’t sufficient.
> The definitions don't change.
Of course they do, what are you talking about? Definitions change all the time with new information. That’s called science.
Definitions don't change. The idea that now that they can it's no longer intelligence is changing. And that's literally moving the goalposts. Read the thread here, go to the bottom part. There are zillions of comments saying this.
You are keen to not trying to understand what the quote is saying. This is not good faith discussion, and it's not going anywhere. We're already miles from where we started. The quote is an observation (and an old one at that) about goalposts moving. If you can't or won't see that, there's no reason to continue this thread.
That is not the argument. The point is that the way you phrased it is ambiguous. “Math” isn’t a single thing, and “cannot” can either mean “cannot yet” or “cannot ever”. I don’t know what the “expert” said since you haven’t provided that information, I’m directly asking you to clarify the meaning of their words (better yet, link to them so we can properly arrive at a consensus).
> Definitions don't change.
Yes they do! All the time!
https://www.merriam-webster.com/wordplay/words-that-used-to-...
> And that's literally moving the goalposts.
Good example. There are no literal goal posts here to be moved. But with the new accepted definition of the words, that’s OK.
> There are zillions of comments saying this.
Saying what, exactly? Please be clear, you keep being ambiguous. The thread barely crossed a couple of hundred comments as of now, there are not “zillions” of comments in agreement of anything.
> You are keen to not trying to understand what the quote is saying. (…) If you can't or won't see that, there's no reason to continue this thread.
Indeed, if you ascribe wrong motivations and put a wall before understanding what someone is arguing, there is indeed no reason to continue the thread. The only wrong part of your assessment is who is doing the thing you’re complaining about.
He seems to be fixated on this notion that humans are static and do not evolve - clearly this is false. What people thought as being a determinant for intelligence also changes as things evolve.
Doing formalized mathematics is as intelligent as multiplying numbers together.
The only reason why it's so hard now is that the standard notation is the equivalent of Roman numerals.
When you start using a sane metalanguage, and not just augmrnted English, to do proofs you gain the same increase in capabilities as going from word equations to algebra.
But the Roman numerals are easy. I was able to use them before 1st grade and I can't touch any "standard notation" to this day.
Proposing and proving something like Gödel's theorem's definitely requires intelligence.
Solving an already proposed problem is just crunching through a large search space.
I think GIT is a negative answer to a problem originally posed by David Hilbert. It was not proposed by Goedel originally. I think Goedel's main new idea was (i) inventing Goedel numbering (ii) using Goedel numbering to show that provability from a finite FOL signature, and a single FOL formula, is reducible to an equation involving primitive recursive functions (iii) devising a method to translate FOL statements about arbitrary primitive recursive functions into statements about only the two primitive recursive functions + and ×.
Later work establishing the field of computability theory (or "recursive function theory" as it was then known) generalised the insights (i) and (ii). In light of that, Goedel's only now-relevant contribution is (iii).
> When will LLM folks realize that automated theorem provers have existed for decades
This is very misinformed. Automated theorem proving was, sadly, mostly a disappointment until LLMs and other Machine Learning techniques came along. Nothing like the article's result was remotely within reach.
You can just about make out those goalposts on the surface of the moon with a good telescope at this point.
How is this not just another proposed problem (albeit with a search space much larger than an Erdos problem's)?
But this isn't a fair bar to hold it to. There are plenty of intelligent people out there, including 99% of professional mathematicians, who never invent new fields of mathematics.
I find it's helpful to avoid conflating the following three topics:
/1/ Is the tool useful?
/2/ At scale, what is the economic opportunity and social/environmental impact?
/3/ Is the tool intelligent?
Casual observation suggests that most people agree on /1/. An LLM can be a useful tool. (Present case: someone found a novel approach to a proof.) So are pocket calculators, personal computers, and portable telephones. None of these tools confers intelligence, although these tools may be used adeptly and intelligently.
For /2/, any level of observation suggests that LLMs offer a notable opportunity and have a social/environmental impact. (Present case: students benefitted in their studies.) A better understanding comes with Time() ... our species is just not good at preparing for risks at scale. The other challenge is that competing interests may see economic opportunities that don't align for social/environmental Good.
Topic /3/ is of course the source of energetic, contentious debate. Any claim of intelligence for a tool has always had a limited application. Even a complex tool like a computer, a modern aircraft, or a guided missile is not "intelligent". These tools are meant to be operated by educated/trained personnel. IBM's Deep Blue and Watson made headlines -- but was defeating humans at games proof of Intelligence?
On this particular point, we should worry seriously about conferring trust and confidence on stochastic software in any context where we expect humans to act responsibly and be fully accountable. No tool, no software system, no corporation has ever provided a guarantee that harm won't ensue. Instead, they hire very smart lawyers.
ChatGPT equalizes intelligence. And that is an attack on their identity. It also exposes their ACTUAL intelligence which is to say most of HN is not too smart.
Citation needed
These types of people need citations for the time of day. They don't know how to debate or discuss in abstract terms. Reality freezes over if no scientific papers exist on the topic.
I don't know man, maybe you got too equalized yet but the things you say are not very smart. Getting angry over that isn't a good argument either.
You probably need to look at your own reading comprehension skills before you comment on my intelligence.
Oh man you have captured the exact emotion I had. These people need randomised control trials to prove any inane thing lmaoo. Reddit brained I tell you
How about I caveat it the way you want:
AI equalizes intelligence in the sense that it closes the gap. Not perfectly, not infinitely, but directionally. The distribution compresses. The floor rises faster than the ceiling, so people who used to be far apart end up operating much closer together.
You can already see it in the Erdős example. The person who wrote that prompt wasn’t some random idiot. It took real cleverness to even set it up that way. But the fact that they could get that far, with assistance, is exactly the point. The distance between “amateur” and “expert” shrinks when the tool fills in large parts of the path.
Now extend that forward. Today it’s one clever person, one problem, one careful interaction. As the tooling improves, that same pattern scales. Better reasoning, better search, better guidance. The amount of lift the tool provides increases, which means the gap continues to narrow.
All the supposed “counterpoints” people bring up are already implied in the claim. “Equalize” here obviously means moving closer to equality. Is it NOT obvious that LLMs don't actually equalize intelligence to a level of 100%? Do I actually need to spell that out? If there was nothing at stake, I wouldn't need to.
But instead people latch onto the most absurd version possible, knock that down, and act like they’ve said something meaningful. It’s the same mindset as that guy demanding a formal paper or citation for an observation you can see unfolding in real time. Not because it’s unclear, but because engaging with the actual claim is uncomfortable. It’s easier to distort it into something extreme and dismiss it than to admit the gap is closing.
More pointedly, I don’t think anyone who opposes AI does so because they want to remain the smart kid in the room.
> If there was nothing at stake, I wouldn't need to
You’re on HN buddy. If you measure stakes by how pedantically you’re challenged, everything will rise to existential terms.
It's more then being the smart kid in the room. The future is pointing to a place where programming is just a one hour tutorial on how to tell AI to do it for you. What happens to you if you're entire identity and career was built on being a programmer as many people are here? THAT is what is at stake.
Yes, I love living in communism too. Imagine if you had to pay money for it or something. The wealthiest people would get unrestricted access to intelligence while the poor none. And the people in the middle would eventually find themselves unable to function without a product they can no longer afford. Chilling, huh? Good thing humans are known for sharing in the benefits of technological progress equally. /s
Before ChatGPT it costs ~$100,000 to aquire intelligence good enough to solve this Erdos problem, now it costs ~$200.
I'm really confused at what you are even taking an issue with.
More specific to HN is the archetype of: "I have spent years honing my craft as a expert programmer, my identity is predicated on being an expert programmer in which high intelligence is causal and associated positively with my identity" That's why ironically most of HN was completely wrong about AI. They were wrong about driverless cars, they claimed vibe coding was trash. It's the people who think (keyword) their stupid/average (aka general public) who got it right... because perceptually they stand to gain from the equalization.
Anyway.. this fear and jealousy is not something most humans can admit to themselves. Nobody will actually be able to realize that these emotions drive there thinking. They have to lie to themselves and rationalize a different reality. That's why you get absurdist takes like this.
To everyone reading. It is obviously that chatGPT does not equalize intelligence to the point of 100%. That statement is obviously not saying that. Everyone knows this. You want proof?:
Look at the declaration of independence... without getting to pedantic: "All Men are created equal" is not saying all Males are 100% equal. Everyone knows this. First off no one is 100% equal.. and second the statement in a modern context is obviously not referring to only men. It is referring to women&men and clearly men and women are nowhere near equal.
So if you all know this about the declaration of independence... how can you not see the same nuance for: "ChatGPT equalizes intelligence."? First ask yourself... do you think you're smart? If you do, then the self delusion I just described is likely happening with you.
What was that about "spreading FUD about unaffordability"?
[1] https://ourworldindata.org/grapher/share-living-with-less-th...
Please show me the steps to get a $200 subscription for free that works 100% of the time regardless of who you are. I'm listening.
You are exaggerating the situation by essentially claiming since some people can’t afford 200 dollars this means ChatGPT is not democratising intelligence. It’s a bit strange to claim this because according to you it only becomes affordable when maximal number of people can afford it. It’s a bit childish.
Directionally it is democratising. Are more people able to afford higher level intelligence? Yes.
It flattened the difference between a top epsilon percentile mathematician and an amateur with money. It didn't flatten the difference between an amateur with a little money and an amateur with a lot of money. It widened it. That's the part I'm scared about.
You are shrugging this off because it currently isn't that expensive. But we're talking about the massively subsidized price here, which is bound to get orders of magnitude higher when the bubble pops. Models are also likely to get much better. If it gets to a point where the only way to obtain exceptionally high intelligence is with an exceptionally high net worth and vice versa, how is that going to democratize anything?
"All men are created equal" is obviously not literally saying all humans are 100% equal. Just like how "ChatGPT equalizes intelligence" is not saying ChatGPT equalizes the intelligence of all humans to a level of 100%.
I'm not going to spell out what I meant by: "ChatGPT equalizes intelligence". You can likely figure it out for yourself, because the problem doesn't have anything to do with your reading comprehension. The problem is more akin to self delusion, you don't want to face reality so you interpret the statement from the most absurdist angle possible.
The admins at HN actually noticed this tendency among people and encoded it into the rules: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."
You seem to be under the misconception that you somehow “own” ChatGPT or are entitled to the insight it provides. You don’t and you aren’t. You are at the mercy of trillion-dollar private companies that owe you nothing. Their products’ intelligence is not your intelligence. Whatever profits you’re seeing from it, it’s currently losing them money. And when that changes, so will your image of them as benefactors of humanity who make intelligence available to all.
> You seem to be under the misconception that you somehow “own” ChatGPT or are entitled to the insight it provides.
Right now for the price of a new car I can definitely get enough hardware to run a local LLM to the quality of ChatGPT at my home. And this is just the status quo. The demand for this technology and the projection of improvement in prices predicts a future where you can run one for the price of a new computer. Wake up.
But who the fuck cares? Point being is AI is equalizing intelligence and you’re just throwing in tangents and side branches to try to disentangle the obvious general truth which I will repeat: AI is fucking equalizing intelligence and if you don’t agree, you’re absurd.
Flagging because this kind of language has no place on HN.
> Then you lecture me about HN guidelines.
Not a lecture. An example of how it's a well known issue. I'm obviously not a rule follower myself; and your content is not really fit for HN either. Once you flag the entire conversation is over. I don't really care, but if I were you I'd rather end the argument by being right instead of running away and tattle tailing to the authorities. Up to you.
maybe the admins come in and block the convo, delete it, and/or ban me. Who knows. I don't care. The fact of the matter is... I'm right, and you know it. Everything I said here is true, and you're turning to this way to end it because you can't face it.
Maybe the models are closer to us than we're comfortable to admit.
I love the confidence that comes from this claim. You can run open models in your laptop today compare to the best models from 2 years ago. But sure spread your FUd about investor money running out
No matter how you spin it, we humans are now becoming thermodynamically less efficient versions of LLMs. We contribute nothing of value to the system, so economics dictates we have no place in it except as investors. Skill is nothing now, and ownership is everything. So yeah, I'm afraid of the future. Call it FUD or whatever, I don't care.
Most people would consider someone who can calculate 56863*2446 instantly in their head to be intelligent. Does that mean pocket calculators are intelligent? The result is the same.
> then they are the one meant to be doing the defining, and to tell us how it can be tested for. If they can't, then there's no reason to pay attention to any of it.
That is the equivalent of responding to criticism with “can you do better?”. One does not need to be a chef (or even know how to cook) to know when food tastes foul. Similarly, one does not need to have a tight definition of “life” to say a dog is alive but a rock isn’t. Definitions evolve all the time when new information arises, and some (like “art”) we haven’t been able to pin down despite centuries of thinking about it.
If you wanted to insist a calculator wasn't intelligent and satisfy my conditions then you can. At the very least you can test for the sort of intelligence that is present in humans but absent from calculators and cleanly separate the two. These are very easy conditions if there is some actual real difference.
>That is the equivalent of responding to criticism with “can you do better?”. One does not need to be a chef (or even know how to cook) to know when food tastes foul.
No it's not, and this is a silly argument. Foul food tastes different. Sometimes it even looks different. You can test for it and satisfy my conditions.
You come across a shiny piece of yellow metal that you think is gold. It looks like gold, feels like gold and tests like gold. Suddenly a strange fellow comes about insisting that it's not actually gold. No, apparently there is a 'fake' gold. You are intrigued so you ask him, "Alright, what exactly is fake gold, and how can I test or tell them apart ?". But this fellow is completely unable to answer either question. What would you say about him ? He's nothing more than a mad man rambling about a distinction he made up in his head.
What I'm asking you to do is incredibly easy and basic with a real distinction. I'm not going to tell you to stop believing in your fake gold, but I am going to tell you I and no one else can be expected to take you seriously.
But you can only do that now, in hindsight. Before calculators, one could argue being able to do math was a sign of intelligence, but once something new comes along which can do math in a non-intelligent way, you can realise “ah, right, my definition was incomplete/incorrect, I need something better”.
> Foul food tastes different.
You’re right, that was a bad example.
> You come across a shiny piece of yellow metal that you think is gold. (…) He's nothing more than a mad man rambling about a distinction he made up in his head.
No, that is not right. Fool’s gold is a thing.
https://en.wikipedia.org/wiki/Pyrite
It’s not the same as gold and you can test for it, but that doesn’t mean you know how to do it. Yet it’s perfectly possible that by being exposed to the real and fake thing you’ll get a feel for each one as there are subtle visual clues. It doesn’t mean you can articulate exactly what those are, yet you’re able to do it.
It’s like tasting two similar beers or sodas. You may be able to identify them by taste and understand they’re difference but be unable to articulate exactly how you know which is which to the point someone else can use your verbal instructions to know the difference. That doesn’t mean the difference isn’t there or that you can’t tell, it just means you haven’t yet found yourself the proper way to extract and impart what you instinctively understood.
No you could always do that. The meaning you take from it is up to you but you could always separate humans and calculators.
>No, that is not right. Fool’s gold is a thing.
I know what fools gold is. I used it for contrast. Fools gold can be tested for.
>but that doesn’t mean you know how to do it.
It doesn't matter. If you claim it exists but you don't know how to do it and you can't point to anyone who can, it's the same as something you made up.
>It’s like tasting two similar beers or sodas. You may be able to identify them by taste and understand they’re difference but be unable to articulate exactly how you know which is which to the point someone else can use your verbal instructions to know the difference.
You are still making the same mistake. Two similar beers or sodas taste different. No one is asking you to come up with a theory for intelligence. All you have to say here is the equivalent of "It tastes different" and let me taste it for myself. But even that much, you can not do. So why on earth should I treat what you say as worth anything ?
> It's the equivalent of nonsensical rambling
I see
Would you consider Socrates not intelligent since he was illiterate and had contempt for the written word?
If you want to classify intelligence, you need to define it properly.
With real general intelligence you'd expect it to solve problems above a certain difficulty with a good clip
I don't doubt that there are many very real and meaningful limitations of these systems that deserve to be called out. But "text generation" isn't doing that work.
Can you refute the argument I made, or do you just want to claim LLMs are drinking all our water?
That's what I thought you meant by "statistical text generator", and is why I was moved to comment.
You managed to include in your blanket and conclusory rebuttal "solving undergrad math problems instantaneously". That was one of my examples because (1) it pertains to the subthread, (2) I was talking about it upthread, and (3) I have direct firsthand knowledge.
As I said elsewhere: I've fed thousands of math problems through ChatGPT (starting with 4o and now with 5.5). They've all been randomized. They do not appear in textbooks. They cover all the ground from late high school trig to university calc III. I do this habitually, every time I work an "interesting" problem, to get critiques on my own work. GPT has been flawless, routinely spotting errors or missed opportunities. If I have any complaint, it's that GPT tends to be too much better than I am at any given point, using concepts from later courses to solve simpler problems.
Square that with the claim you're making.
I can do the same thing with vulnerability research (I've been a vuln researcher since 1996 and I use LLMs to find vulnerabilities). But this thread is about math, and it's even easier to show you're wrong in the context of math.
"A farmer has 17 sheep. 9 ran away. He then bought enough to double what he had. His neighbor, who had 4 dogs and 14 sheep, gave him one-third of her animals. The farmer sold 5 sheep on Monday and again the next day, which was Wednesday. Each sheep weighs about 150 lbs. How many sheep does the farmer have?"
He bought enough to double what he had: 8 more sheep, so 16 sheep
Neighbor has 4 dogs + 14 sheep = 18 animals
One-third of her animals = 6 animals
But the problem does not say all 6 were sheep. It says “animals.” So the exact sheep count depends on which animals she gave him.
Then:
16 + s sheep from neighbor - 5 - 5 = 6+s
where s is the number of sheep among the 6 animals she gave him.
So the answer is not uniquely determined.
Possible sheep count: 6 to 12 sheep, depending on whether the neighbor gave him 0 to 6 sheep.
(I clipped the GPT5 answer here, but will note additionally that even the LLM built into the Google search results page handles this question; both note the possible trick question with the days of the week.)
By your logic, the only "correct" answer for an LLM to give to this is "the person who asked you this is fucking with you, this is not a real question". I concede: this is a limitation of modern LLMs: they will try to answer stupid questions.
Obviously, they can do math.
Again if you want to say they're limited in some way, I'm all ears, I'm sure they are. But none of that has anything to do with "statistical text generation". Apparently, a huge chunk of all knowledge work is "statistical text generation". I choose to draw from that the conclusion that the "text generation" part of this is not interesting.
You seem to be making the claim that LLMs are statistical text generators, but statistical text generation is good enough to succeed in certain cases. Those are different arguments. What do you actually believe? Are we even in disagreement?
Umm, why doesn't it capture it? Why can't a statistical text generator do amazing things without _actually_ being intelligent (I'm thinking agency here)? I think it's important to remind ourselves, these things do not reflect or understand what they're outputting. That is 100% evident with the continuing issues with them outputting nonsense along with their apparently insightful output. The article itself said the output was poor but the student noticed something about it that sparked an idea and he followed that lead.
To clarify: the problem I have with "statistical text generator" isn't the word "statistical". It's "text generator". It's been two years now since that stopped being a reasonable way to completely encapsulate what these systems do. The models themselves are now run iteratively, with an initial human-defined prompt cascading into series of LLM-generated interim prompts and tool calls. That process is not purely, or even primarily, one of "text generation"; it's bidirectional, and involves deep implicit searches.
To be clear, I'm 100% with you that "next token predictor" is stupid to call what these machines are now. We are engineers and can shape the capability landscape to give rise to a ton of emergent behavior. It's kind of amazing. In that sense, being precise about what's going on, rather than being essentialist (technically, yes, the 'actual' algorithm, whatever that even means, is text prediction), is just good epistemology.
I still think it's still a very interesting question though to ask about deeper emergent structures. To me, this is evidence of a more embedded cognition kind of theory of intelligence (admittedly this is not very precise). But IDK how into philosophy you are.
So you agree that LLMs are in fact statistical text generators but you don’t like people use that fact in arguments about the capabilities of the things?
But it is no longer useful to bring that fact up when conversing about their capabilities. Saying "well it's a statistical text generator so ..." is approximately as useful as saying "well it's made of atoms so ...". There are probably some very niche circumstances under which statements of each of those forms is useful but by and large they are not and you can safely ignore anyone who utters them.
And your evidence that they aren't is ... ?
I do think LLM's are evolving towards this kind of embodied cognition type intelligence, in virtue of how well they interoperate with text. I mean, you don't need to "make the text intelligible" to the LLM, the LLM just understands all kinds of garbage you throw at it.
Now the question is: Is intelligence being able to interoperate?
In the traditional sense, no. Well, in a loose sense, yes, because people would've said that intelligence is the ability to do anything, but that's not a useful category (otherwise, traditional computer programs would be "intelligent"). But when I hear that, I think something like "The models can represent an objective reality well, it makes correct predictions more often than not, it's one of these fictional characters that gets everything and anything right". This is how it's framed in a lot of pop culture, and a lot of "rationalist" (lesswrong) style spaces.
But if LLM's can understand a ton of unstructured intent and interoperate with all of our software tools pretty damn well... I mean, I would not call that "a bunch of hacks". In some sense, this is an appeal to the embedded cognition program. Brain in a vat approach to intelligence fails.
But it clearly enables new capabilities that previously were only possible with human intelligence. In a very blatant negative form: The surveillance state is 100% now possible with AI. It doesn't take deep knowledge of Quantum Physics to implement, with a large amount of engineering effort, data pipelines and data lakes, and to have LLM's spread out throughout the system, monitoring victims.
So I'd call it intelligence, but with a qualifier to not slip between slippery slopes. It may even be valid to call the previous notion of intelligence a bad one, sure. But I think the issue you may be running into is that it feels like people are conflating all sorts of notions of intelligence.
Now, you can add an ad hoc hypothesis here: In order to interoperate, you have to reason over some kind of hidden latent space that no human was able to do before. Being able to interoperate is not orthogonal to general intelligence - it could be argued that intelligence is interoperation.
If you're arguing for embodied cognition, fine, we agree to some extent :)
The fear is that the AI clearly must be able to emulate, internally, a latent space that reflects some "objective notion of reality". If it did that, then shit, this just breaks all of the victories of empiricism, man. Tell me about a language model that can just sit in a vat, and objectively derive quantum mechanics by just thinking about it really hard, with only data from before the 1900s.
I don't think you need to be this caricature of intelligence to be intelligent, is what I'm saying, and interoperability is definitely a big aspect of intelligence.