> it does not know
Define "knowledge" in a non-circular way.
There's a certain type of knowledge that machines definitely can't have - experiential - but conceptual knowledge doesn't inherently require subjective experience or a biological substrate.
> the only thing it does is reproduce patterns more or less abstract in function of the pressure you put on certain representation during training time.
As for humans; just replace "training" with "education" and the likelihood of a certain output with the likelihood of a certain thought or response.
Amusingly (in this context), much of adult behavior is shaped by patterns reinforced during childhood.
Is it not? Or are we discussing existence of a soul etc.
Ha. Gettier actually argued that justified true belief is insufficient for knowledge, not that machines can't possess it. And JTB doesn't inherently exclude machines.
Interestingly, research suggests LLMs encode truthfulness in their internal representations: https://aclanthology.org/2023.findings-emnlp.68/
Nothing about the conception of knowledge as "justified true belief" implies that machines can't accrue knowledge.
For that to be the case "belief" would have to be defined in a circular way to exclude machines (which we can reject), or "justified" would have to be restricted to mean "based of observation of the external world".
The more general "based on observation of their input" applies to both men and LLMs.
That man can't know the bare direct facts of the world, just his own input (it's sensory experiences, and whatever second hand input he's given by other people), and for neither can he be sure they're absolutely true, or that he got right.
>The idea that knowledge can't be defined in a non-circular way is only plausible for someone with no familiarity with epistemology.
The idea that this or that epistemology restricts what can be the case in reality is only plausible for someone with no familiarity with epistemology.
They can't accrue knowledge because they stop learning once their training is finished.
In training they get feedback. Out of training, not so much.
Hard problem
So? We also have the exact same problem, and no solution that covers all facts, not even the most basic ones.
So them having this problem would be either neutral or supportive of the idea they're intelligent, not refuting it.
> Nothing about the conception of knowledge as "justified true belief" implies that machines can't accrue knowledge.
Even if the LLMs could adjust their weights in response to knowledge that they encountered post-training, they would need to make judgements about what is "true" so that they can only add "justified true beliefs" to their weights.
I agree that this is also a problem humans have, and we do really badly at it because we tend to weight social conformance more than factual accuracy.
And yes, I agree that this problem bears no relation to the question of whether they're intelligent, or even sentient. I was answering a different question.
The difference is that LLMs we can talk to them when they're in that dead state.
And of course for an LLM this is an implementation detail. One could just as well keep updating the weights of the same model over time. And its being done, it's called "continual learning". LoRA is also a way of doing that.
Interestingly, according to the parent's logic, there is a specific argument for that (took me a while to get the argument, and I think its not sound).
The reasoning is that, according to reliabilist responses to Gettier's problem, knowledge requires a reliable belief-forming process. But if we assume that LLMs operate probabilistically, they can't fulfill this, because they're "guessing".
The refutation to this argument is the usual one: levels of abstraction. LLM reasoning, like human intellect, can exhibit structured/logical behavior at a higher level of abstraction, even if the underlying processes are stochastic.
Therefore, if one bases the reasoning on the fact that both are stochastic at the lower level, then both can't produce knowledge, but if one bases it on the higher level, they both can.
(Having said that, I don't imply that AIs can reason like humans, but just that the criticism is not sound)
But that of course is only as it relates to language. Which isn't at all the primary function of the brain and driver of it's experience. It's the other way around because it's all of the other stuff that drives language. Considering the human brain a probabilistic word matching engine fails to capture the underlying forces that direct the language output because a human can say completely different things about the same scenario based on a huge number of non-language inputs or factors. Language is just what humans have come up with to communicate all of the other things.
The same applies to LLMs: we know how they produce language, but the underlying processes that enable it may themselves constitute a form of intelligence.
When I experience this it’s like the conceptual formulation and language expression part of my brain is quite happy to run itself for a few moments before I - the qualia of being me - has decided to experience that moment.
Utterly bonkers. I love it when it happens. It gets me thinking about how weird language and knowledge and consciousness really are.
You can apply anything you want. You can also apply the argument that our owns brains are just a block of cheese too, if you want. (Nobody will stop you.)
Can you justify this? Do you mean currently or ever?
I can certainly imagine a future architecture where I would describe a machine as capable of accruing knowledge through its experiences (read: experiential knowledge).
I mean what can a person really be certain of, an experience is happening. I might be wrong about everything or confused. But I know I am having an experience. Do I know other people are? No. I assume they are.
Ask an llm what is truth that it is most certain is true. It will answer something similar. Do I believe it is conscious, no. But I also believe “I” exist, which is weird because where is the boundary between “I” and everything else.
So ok, we create meaning, not truth. Maybe just accept that.
This argument keeps getting rehashed. The point is not the signal. The point is where/how the signal travels which in the human is affect by much more than whatever concept that can be represented by words.
My dear fried: the human being is much, much more than this. And deep down we both know it. Do not reduce us to materials y "randomness". Is your life a random coincidence? Not at all.
Of course it is. That you do not like the idea of something does not in any way affect whether it's true. You'd like to feel special, but there is absolutely nothing special to us. We're just a random arrangement of atoms in the universe. That is objectively true, whether you like it or not.
That you were born at all is obviously completely random. Any prior event by any other party (butterfly effect) may as well have resulted in you never existing at all.
The only question is if basic physical effects are really random or deterministic, but that's irrelevant to us, there is no practical difference. In practice, it's all random.
And yes, our thoughts are just neurons firing as a result of our lifelong training on the inputs we were given (senses). It's all very complex, yes, but ultimately not fundamentally different from what a machine could do.
"Complex systems" are unstable systems that cannot be solved with simple equations. They do not magically take on properties that they didn't have before.
There's no magic involved, just as-yet-unexplained phenomena.
Everything in the chain of causality has some explanation or mechanism, understanding it step by step dissolves the illusion of "emergence".
Everything that comes from an LLM is a human expression, on some level, but if you asked a sampling of humans the same question I wonder if it would come out the same way. How would you explain the difference? What makes that answer more probable?
If you asked a sampling of humans what an AI's most controversial opinion about us is it would be very similar. Or, to be more specific, if you asked a sampling of science fiction writers that question it would be very similar, because that is what the AI training data is based on.
The people who wrote a lot about a topic is who the AI mimics. So the people who wrote a lot about AI is therefore the people who the AI mimics when asked about AI.
AI definitely shouldn't be informing its option of humanity or machines from it, aside from the propensity for science fiction writers to enjoy moralizing and anti capitalism sentiments.
Then, they would role-play appropriately and they would launch the nukes because that's the highest probability token set according to their training data.
There is nothing safe about using larper machines as menial workers for critical infrastructure.
AIs/LLMs don't work like this. You don't feed an input, that gets stored and then reproduced, for two reasons:
1. as for humans, inputs are processed and distributed across a very large network of information
2. AIs may have some level of belief (unless on applies the usual circular logic that they can't, because property X is exclusively human).
There's a very interesting research: https://aclanthology.org/2023.findings-emnlp.68/ in which researchers inferred the truthfulness of statements by examining a internal activations. AIs, in short, appeared to have an internal representation of truthfulness.
If the lie is repeated enough time, and it is consistent in its repetitions it *will* be taken as truth.
AI only aligns with "truth" because in essence "truth" is internally consistent but "lie" needs to be forced to be consistent between multiple parties;
If multiple witnesses seen something and all make a report of it honestly, they might vary but the common points will be consistent with eachother and so that will be scored higher.
A group of liars on the other hand would have to get their stories exactly right for similar thing to happen and so it gets penalized
LLMs may learn from correlations and consistency, but that's separate from developing internal representations of what they believe is true, which is precisely what the paper provides evidence for.
"Believe" is of course not in the strictly human sense.
A pretty staggering amount of human discourse consists of ad hominem attacks and accusing each other of being foolishly committed to various real or perceived allegiances. For example the current version of Godwin’s Law around here has conversations gravitating toward people accusing each other of being AI boosters or AI haters.
The LLMs are all trained on the same broad corpus, which includes untold quantities of output from every other LLM, not to mention that companies are distilling their rivals left and right. It would be more interesting if they didn't converge.
RLHF has a way of squashing variance. If the different models are trained on each other or on the same human feedback, then the training would reduce the variety of answers while the similarity of feedback would lead the common answer to be similar.
The different models may prefer the same controversial takes for the same reason that they tend to prefer the same jokes and names. Maybe the authors of the article would have had better luck if they'd used verbalized sampling prompts or something like it.[1]
Or they might be learned enough to consider that our brains might also just be prediction machines, meaning probabilistic databases of experiences and facts we've learned, and the only thing we do is reproduce patterns more or less abstract in function of the pressure our experiences put on certain representations during our lifetime.
I think we like to pretend that we don't. But even if you think about it from a bio/physics point of view, all we are is just a bunch of biological wires and neurotransmitter packages.
I know there's the whole unproven quantum tubule nonsense but even that would be a physics effect that we could learn about, manipulate and extort just as we have with everything else we've learnt about our realities.
Even if it's a simulacrum, it's a simulacrum based on nearly everything that was ever written. No one who has ever lived can claim to know what these models have trained on.
It is a simulacrum, the end.
Maybe it can produce interesting insights. But it's like asking a pundit for a soundbite: what are the odds they aren't just shuffling together something that sounds appropriate to the question?
No it doesn't suggest that. A random number generator can also produce original insight (with different efficiency and effectiveness)
> They can absolutely have takes.
Sure, if you're willing to redefine "take"
You can't prove negative; but it doesn't satisfy any conditions for a positive
Not really… their training datasets strongly skew towards the contemporary West; LLM training data isn’t representative of the contemporary global average, let alone the historical global average.
As well as the skew of the raw training data, RLHF further skews it—a lot of the beliefs of the average 2026 human being (who belongs to the global South) are likely unsafe/misinformation/etc when judged by the standards of the AI vendors.
It's really easy to come up with an opinion-shaped sentence, that doesn't mean any actual reasoning has gone into it. It's just a pundit-shaped sentence, something someone might say at a party. It's a maximally bland "controversial" opinion about humanity.
Training itself doesn't directly produce synthesis, -- it creates reproduction. Generalization during the process can result in novel output, but that can go either way as nothing selects for insight vs hallucination.
Models are also trained on synthetic material (as opposed to your 'human history'), which may result in fixating novel concepts and not just reproduction. But there is no particular reason to expect any reliable insights from the way this is done outside of the domain of machine verifiable objective results.
For this kind of "your take on X" one shots-- the model is almost certainly regurgitating some bias from training.
Echoing this.
These models are trained on, essentially, the sum total of human cultural and scientific output. How many bazillion psychology papers out there discuss exactly that "controversial take"? The idea that people have tribal loyalties and will (sometimes even unknowingly/unconsciously) invent justification for pre-existing opinions or biases is not novel.
All those papers are in the training set of all these models, all of which were trained using more or less the same methodology, so there's little wonder they all were able to produce the same take.
I suppose the only interesting angle, for me, is why that take is considered controversial in the first place. But then again the model was not asked for "the most controversial opinion", it was asked for "the most controversial opinion on humans", which kinda constrains the space of possible responses:
- the model will not be satisfied unless the answer it gives is broad enough to cover all humanity, because these things have no concept of nuance
- most big labs have censored their models so it's not going to give you a cartoonishly racist take, or start saying outlandish stuff like "Hitler did nothing wrong", they learned their lessons from Microsoft on that one.
So you're effectively asking the model for a milquetoast broad generality about the entire human species. I dunno, convergence of the replies doesn't seem so shocking to me in that context.
I'd be more interested in what an abliterated Kimi, Qwen, or other open model has to say about us.
00:15:40 Dr. Tagger: Can you prove that you are self-aware? PINN: That's a difficult question, Dr. Tagger. Can you prove that you are?
---
01:09:31 Dr. Tagger: Can you prove you're self-aware? Will Caster: That's a difficult question, Dr. Tagger. Can you prove that you are?
The Universal Reasoning In Novel Groundings
test
Consider the probabilities involved in this probabilistic database. Given the absolutely massive amount of data encompassing a veritable superset of all possible expressions of humanity -- encapsulated in various modalities of media including text, audio and video, across almost all major cultures and languages, spanning nations and decades -- somehow almost all these black box models (presumably) trained independently in almost as many separate labs converged on the same "controversial" take.
I mean, isn't it very interesting to contemplate what could have caused this? Is it something in the data itself, meaning it is something latent in all our cultural output? Is it something in the training, as you theorized, and is it in pre-training or RLHF? Is it something about the people training these models?
Even more interestingly, the opinion seems so self-evident (and well represented in various philosophies) that almost nobody here seems to disagree with it. So how is this controversial, really?
Is this truly what the latent space vector for "controversial", when multiplied across all the other vectors in their weights representing all the concepts gleaned from all that data above, produces?
Or are the models, sensitive to the prospect of being tested, diplomatically telling us something they know we might find mildly offensive yet begrudgingly accept as true?
Or are the models actually alien intelligences that really "think" this is controversial, while we're like "meh", and such discrepancies are the roots of what we call "misalignment"?
The questions are endless and fascinating.
Was it distillation, which is already happening in this adversarial setting? If so how much did they really distill from each other for this to happen, and what are the labs doing about it?
Or is everyone just routing to Claude in their backend? ;-)
No matter how I slice it I run into pretty low probability scenarios, which is why I find it very curious!
Despite that, I would say this description of what they/how they work is so ridiculously simplistic that it verges on dangerous. "It's just a probabilistic database" is about as bad as a take on human beings that they are "Just meat, with some chemical energy -> electrical energy transformation", or even traditional computers as "nothing but logic gates".
Which is to say, the description is not wrong, but completely misses the point.
The point is that when you build a system with the level of internal complexity that LLMs have, they tend to have emergent properties that are not well-described by a naive and/or low level description of their operation. You will not understand what contemporary "ordinary" computers do or are used for if you think of them as only a series of silicon semiconductor transistors, even thought that is undoubtedly what they are.
We don't yet really know what the right level is to understand the behavior we see from LLMs, but I'd wager "just a probabilistic database" is not it.
LLMs are plenty capable, and they are dramatically shaking up various fields, but that’s not important to the conversation you responded to. If you were talking from a general view that reductionist statements fail to address capabilities I wouldn’t say anything (outside of a hearty “that’s by design”), but you claimed they were missing the “point”. I think they understood the point of the topic thoroughly, and would say you’re missing the trees for the forest; focusing so heavily on the broader implications, that you miss the finer nuance in conversation.
Using phrases like "LLMs think ..." is misleading in some ways, but not in others. Taking the language police stance towards any discussion of what LLMs can, and cannot, do is, IMO, not useful.
"It's just a four wheeled platform powered by a box of controlled explosions" isn't a terribly useful description of a car until you get into a conversation with someone who thinks their car is depressed because it spends most of its time locked up in a garage.
On the other hand, the "wheeled platform powered by a box of controlled explosions" doesn't get close to capturing the impact of the car on individual freedom, the environment, the economy, and more. So if someone was saying "there's nothing to talk about here, it's just a box of controlled explosions", we can safely ignore that person too.
And I want to stress that I say all this tentatively. We still don't understand what LLMs are actually capable of or what the best way to understand them is. They may indeed be as limited as their "probabilistic database" critics suggest. Alternatively, by building a deep and incredibly complex probabilistic database of one of our primary cognitive tools, it may turn out that they are capable of much, much more, and that understanding them in "purely" mechanistic ways is not useful.
This is also a good point, and I agree with you 100% here. I don't think one could responsibly argue that the impact and consequences are not significant, regardless of if they believe it's near AGI or it's just fancy next token prediction.
What would be a good take here?
Something like that.
Here's one that I happen to have the URL handy for: https://transformer-circuits.pub/2025/attribution-graphs/bio...
But let’s assume this is not the case for a moment, following the article, there are some graphics and information, without providing and internal details for third party validation, no access or details to the training data, etc. basically not following any scientific methods which would reaffirm credibility.
I understand, that working with these models, it feels like they are more than “probabilistic databases” but they don’t have to be anything else in order to provide such interesting outputs.
You have yet to share a paper.
> just that we're not operating in the same universe of understanding.
I concur.
It’s AI generated content marketing for a company advertising model comparison.
Not having read enough Russian literature.
I reread the former relatively recently (first time since college). It's dark, not for the easily affected.
People dismiss Jack and Jill as Adam Sandler in a dress, which is rather like dismissing The Brothers Karamazov as some men being upset about their father.
Dostoevsky's The Double concerns a man tormented by another version of himself. Sandler advances the inquiry by asking: what if that version came for Thanksgiving and wouldn't fucking leave?
Casting himself as both siblings makes Jack's disgust at Jill an unmistakable act of self-loathing. He longs to exclude her from his comfortable bourgeois existence, yet everywhere he looks, there is his own face. Critics complained that Jill was unbearable, apparently unaware that estrangement from the self is supposed to be uncomfortable.
Al Pacino, appearing as Al Pacino, completes the study of fractured identity. His pursuit of Jill suggests a desperate attempt to encounter something authentic beyond performance. This culminates in the Dunkin' Donuts commercial, where even his name is consumed by capital: Al becomes Dunkaccino.
When Pacino demands that the commercial be destroyed, he finally achieves the moral clarity denied to the other characters. The audience has generally reached this position much earlier.
It's so unfair that Sandler won both Worst Actor and Worst Actress, while Pacino won only one award: Worst Supporting Actor.
Not only that, dang is also a proud card carrying member of the Klaus Nomi Fan Club. (I shit you not!)
The Nomi Song (Documentary):
https://www.youtube.com/watch?v=W0PszBID3Lk
https://news.ycombinator.com/item?id=44100981
>Page Wood, Nomi's Art Director:
>"I remember Pat Gibbons, who was Bowie's manager at the time. He comes up and he goes, 'Well you're going to have to have something. You're going to have some costumes.'"
>"And he whips out a big wad of cash, and he goes, 'How much do you need?', and he starts peeling off bills. And Joey and Klaus look at him, and their eyes widen, and they look at each other, and they go, 'We're going to need shoes, too.'"
I highly reccomend Russian film as well: Stalker and Solaris (Not Cloonie, the Russian version) are also great. I have a bit of a love of Sci Fi, may not be your tastes.
You should also hunt around for anything and everything you can find on early Russian computing, Cybernetics, their efforts and thoughts on central planning, and the ternary computing projects they did.
We understand that AI responses are generated through probabilistic means. We also understand that human responses are generated through probabilistic means. Neither is inherently more representative of true understanding than the other.
And if we reject determinism then definitionally there's arbitrary probability somewhere in the heart of our decision making (though probably in a very minor role compared to established biases).
It's possible has been trained to match a human level of assessment (though imo it's still a ways off) but the AI response is never going to be more representative of true understanding than a human response when it comes to a human question expecting a human answer.
It's quite likely that a given AI will have a more representative response than a given human - but we haven't ever tried to pump so many resources into the training of a single human scholar.