Hacker News Reader

Comment by NaiveEmpiricist | Hacker News Reader

Norvig (and Chomsky) are talking at cross purposes.

Norvig confuses engineering for science. If an engineer wants something that works (and guarantees that it will work) then you use the most effective tool available to achieve that goal. No problem there. But there's absolutely no reason to think that tells you anything scientific about the phenomenon you're modeling. For example, aeronautical engineering says very little about how birds fly. And - currently relevant - LLMs learn to use and understand language, but they use millions of times more training data, and millions of times more computing power, than a child does. So an LLM may be a great technology, but it's probably not a good model for how humans learn and use language.

Chomsky OTOH doesn't see any use for statistics in science, at least not in linguistics. Fortunately for the field of linguistics, Chomsky's influence in linguistics is waning and linguists are beginning to use statistical tools to do science.

View on HN

> Norvig confuses engineering for science

I don’t think Norvig is confused. He explicitly addresses this point right at the top of the article.

His argument is that successfully engineering a system using a statistical method provides some bits of evidence for the hypothesis that the system itself is statistical.

Which is obviously true, so long as we take care to weigh evidence properly.

> there's absolutely no reason to think that tells you anything scientific about the phenomenon you're modeling

No, this is not how evidence works. If you can accurately model a system with method X and you cannot come up with a model that works with method Y, you should think it’s more likely (but not certain) that X is the actual physical model.

And indeed modern cognitive science (see Predictive Processing) also has a bunch of evidence for the brain as a hierarchical predictor, rather than a rule engine.

To your points - planes are not model organisms of birds, but they do shed insight on the bounds that birds must obey, eg thrust-to-weight etc. And the sample efficiency point really needs to die. The human brain is born with an extremely complex prior baked in. An LLM begins pretraining with almost nothing (just a very simple architecture and a few simple algorithms like attention), and has to learn the basic architecture of cognition from scratch. A better baseline would be how many samples were processed by every animal in our genetic ancestry that had a brain, since brains existed.

> No, this is not how evidence works. If you can accurately model a system with method X and you cannot come up with a model that works with method Y, you should think it’s more likely (but not certain) that X is the actual physical model.

Could you expand on this point?

Try to model radioactive decay via a process other than statistical and you will almost certainly fail, since that is our best understanding of it.

I was also going to suggest coin flips, but it’s not quite the same… I think any chaotic system is generally best modeled that way, and if you cared about the effect the system has on other things, it’s unlikely you can do better since measuring the initial state will not be feasible.

Try to express it in anything but the language of stats you will fail because we haven't invented another language for it; we could but have no reason to with the ubiquitous nature of stats syntax and semantics

Math is not physics and physics is not math. Writing a Superman story does not mean the language and pictures represent physics

That's the difference being highlighted here; stats are not evidence of gravity and electromagnetism; physical phenomena when materials influenced by those forces are relative to local gravity, electromagnetic fields are our evidence

Stats of 1960s demographics did not capture immutable physical phenomena because entropy-> generational churn -> those people are now dead

The languages do not matter; they're just to communicate some ground truth.

Shackling society to the past through careful statistical methods relative to finance and politics to avoid societal change is not a scientific conclusion; there's no reason to use 30-40 years ago as the center of our social universe

The concern with stats consumption is we walk away from a result as if its an immutable forever conclusion as we ignore that gravity and electromagnetism, etc have not stopped changing ground truth; past truths that no longer matter to our next choices

> Writing a Superman story does not mean the language and pictures represent physics

You can write many imagined stories, but one of them will match observed reality. It won't be about an airborne male model who wears his underpants on the outside. But you can write a story that describes reality. And you can describe observable phenomena with statistics. Of course, you can describe plenty of pseudoscientific drivel with statistics too.

The key point is quite simple: You can write any retroactive explanation you want. But only some stories are _predictive_. And you can weigh models by how good their predictions are; which was my original point above.

i agree. but it's uncanny that stochastic generation of language yields a form of recognizable intelligence.

in college, i was somewhat involved in cogsci wrt semantic mapping of linguistic concepts based on human surveys (this was well before LLM/AI-word2vec), and it surprises me today how the mathematical properties of relatedness/distance seem now to drive machine intelligence, something which we did not consider at the time at all.

maybe i'm getting in my metaphysical sentimental stage, a turn a lot of adults make after middle age, but i wouldn't fully dismiss the fact that there may be an equivalence.

Indeed. Who'd have thought that mere statistics could do that? Hat's off to them - they made the darn thing work.

There's pretty interesting (I think) research in how effectively it's possible to scale down LLMs. If you're interested, take a look at the BabyLM conferences:

https://babylm.github.io/

(I have no connection to them.) The work there demonstrates that gap between what we can do now with human-relevant amounts of training text, and what remains to be discovered. Quite a bit, it looks like.

“Mere statistics” is heavily underselling what the transformer architecture does. Unless you think “mere ion channels” is a sensible way to think about the human brain.

"Mere statistics" is how physical reality works; I don't mean the language, the syntax

I mean shifting influence of gravity relative to an object with enough mass type ratios are everywhere

The language of math started as visual art to depict such ratios; in modern times this ground truth associations is frequently overlooked or never understood to exist in the first place without engagement

Differential diagnosis in medicine is exactly the same process; such processes and their successes are everywhere.

The issue we have in software is the outcome isn't as obvious as jabbing a blood draw needle in a patients eye; we spent a lot of time wasting time on illusory needs

The existence of LLMs is telling us something, surely it means something. I'm not sure we've quite figured it out yet but its probably something along the lines of 'intelligence might not be as complicated as we thought' ... a fourth narcissistic blow to humanity after Copernicus, Darwin and Freud.

The narcissism might not be in thinking 'intelligence might not be as complicated as we thought’ - the narcissism might be in thinking ‘LLM intelligence is the way we thought’.

I understand the charge, but it's soemwhat unfair. It's more like we start to see a correspondence which even 20 years ago was barely considered

It's also not the first time a breakthrough technology was compared to the human brain/mind, but we also know biologically animal nervous systems are not digital computers, telephone/telegraph networks or hydraulic systems.

I'm not saying LLMs work in the same way as us. But a few years ago humans used to be the only competent users of language on the planet. Now its humans + LLMs who are competent users of langauge. This is a big change, and I'm saying that it might be pointing to something deeper about complexity and how approaches to deal with complexity arise in the universe.

sure, but its facile to dismiss these breakthroughs as naive metaphors. physicans do understand now many bodily systems can be modeled with the same tools an engineer will model his machines. its not because these abstractions are incomplete, they are wrong.

We train them on the sum of human knowledge available in digital format, so I'm not sure that's an indictment, and the large models are incredibly complicated to the point it's hard to reverse engineer how they work so well and what they're actually doing to build internal models to predict the next token. Several billion to several trillion parameters is anything but simple.

Right, we needed compute on a large and fast scale to make LLMs actually work. But the basic principles are quite simple: train on a huge dataset for long enough, using some fairly simple ML maths but across a very large dimensional space, and then out the other side you get something that can competently use language, and is capable of doing useful work. This is somewhat surprising, no?

Yes, it is surprising.

Freud doesn't belong in that group. Freud was a fraud.

> stochastic generation of language yields a form of recognizable intelligence.

Can you use a human language well enough and it not sound intelligent? And no, I don't mean just syntactically correct because semantics is a part of human languages too. I think the astounding thing is that they managed to model a language this well. It's like trying to learn English without a functional understanding of the word "Please".

My take is that a lot of the intelligence we natively ascribe to humans exist in human languages and just like the nutrients in your food, by harnessing a language that possesses latent intelligence, you too can behave like an intelligent person. And how did the nutrients of intellect got in there in the first place? Just like languages evolve nowadays: via contributions from speakers over time. Not unlike soil fertilisation.

And language itself is so very 'stochastic' too, due to complex subjectivity. Every word floats on a variable sea of individual choice.

For example, if I compose your post as :

"Good prose embodies intellect gradually evolved by myriad humans over time, so it wows me to see LLMs create it, though I soon realised that the intelligence is not being created by the LLM but rather is already present in the human-evolved input datasets - the prose itself! - which they statistically arrange."

..the particulars of the language are entirely different (and omit some of your chosen nuance) but most readers would receive the same "core truth" from it, the same measure of beneficial information.

Which highlights the great fluidity of language; that is another of the key underlying mechanisms allowing LLMs to flourish.

I think there is a distinction to draw there between stochasticity in the use and in the evolution of language (which is not clean as they are not independent). The language evolves stochastically, as it does so by being used in exchanges. But the use could be deterministic with the exchanges being random and the function of update based on exchange. I still think there is stochasticity in the use itself, I choose my words, go back and delete and rewrite, explain again in a different way when the audience looks at me unimpressed etc. I just don’t see fluidity as a proof of stochasticity of evolution or the other way around and in fact think the interaction is a quite interesting domain.

I present this as by-no-means a complete argument, but a very salient point imo is the notion of the 'accent'. It is the most prominent etymology mutation in language (driving inter-language variance between adjacent cultures), and exists not only in the human but in the animal kingdom. Its basis? The maxillofacial expression of the most dominant speaker(s) within a group, which shapes their vocal inflection and passes to the wider culture by social effect.

Regardless, the fluidity of language is self-evident to an extent : it is partly embodied in the entire concept we call figures-of-speech (a linguistic phrasing can be literal, or sarcastic, or described by metaphor, ...). Partly in the common reality that our dictionaries list multiple possible definitions for many words. Partly in the fact that (many) synonyms exist. Partly in sheer human individuality.

> i agree. but it's uncanny that stochastic generation of language yields a form of recognizable intelligence.

It's funny, I used to think it was complete and utter horseshit when literature professors would say that the meaning of a text is created by the reader, not the author.

Since LLMs have become commonplace, I have completely reversed my opinion on that.

That you see recognizable intelligence in the output of stochastic generation is because you are imposing meaning on it. It is why "claudish" is so hard to read--you are trying to impose meaning on a sequence of words that was stochastically generated, and it takes mental effort to do that.

I fail to see what exactly is explained, or clarified by the theory here. Postmodernist philosophers are notorious for writing texts which make readers struggle. Surely, it doesn't mean "Foucault-ese" is somehow more stochastically generated than "Claudese".

  > you see recognizable intelligence in the output of stochastic generation is because you are imposing meaning on it

i am not an expert in any of this, but intuitively i largely feel the same... the character we see as "claude" is mostly in our heads; its almost like a "super rubber duck" phenomenon or something

I'm not so sure about that.

If Claude can write formal mathematical proofs (that a machine can check), or it can write software that works, where's the head of the reader that imposes the meaning?

Reading what you wrote, I must interpretively attempt to construct a model of what you may have meant and what you may believe, because on the surface it contains multiple claims that are frankly false in my experience (itself a probabilistic world model).

> a sequence of words that was stochastically generated

As was your comment (and mine, and the others on this page and elsewhere).

With Claude I know that there is no source meaning or belief, but it helps to impose Dennett's "intentional stance" and treat it as if there were, up to a point. That there is such a "point" allows be to manipulate it in my attempts to extract factual information implicit in its training data, expressed probabilistically via its stochastic processes.

> it's uncanny that stochastic generation of language yields a form of recognizable intelligence.

It most certainly does not. It's obvious to anyone who critically examines LLMs, without letting the hype override sense, that there's zero intelligence there.

To the cynic, Chomskyism is exactly what Thomas Kuhn warned you about "normal science".

That is, you can apply for grants and write papers in the transformational grammar paradigm endlessly. You can pose and answer questions that make sense within in the system.

Chomskyism was disappointing in two respects: (1) was that it was an inadequate basis for machine understanding of text, (2) was that structuralists were hoping everything in culture could be understood in terms of "... having the structure of language" but syntactic structures weren't all adequate for that task.

(1) and (2) both pointed to Chomskyism being hermetic and not something really useful or inspiring to other fields and maybe just a little less "true".

> You can pose and answer questions that make sense within in the system.

Yes this is one of my critiques of the academic system (I have finished a PhD and will hopefully never work at a university ever again). Many papers in the introduction start with explaining how the study builds on plenty of previous research, but that alone doesn’t prove that something is a good direction to investigate. If academics were completely honest, many papers would write “We researched topic X because we got funding for it. But since we actually don’t care about X, we just wrote a proposal about our favorite niche topic that only one of the co-authors cares about, namely topic Y, and wrote it such that it sounds like X. Next we show that Y is actually very cool in the hope to convince others that it is cool plus to convince others of the conclusion that our most senior professor already found during the PhD which was 30 years ago. Apart from the professor, we don’t believe in the main argument of the paper but since he got the grant we’ll just go with it in the hope to one day be able to push our own ideas forward”.

Actually, there have been a number of rounds of paradigm shift (or attempted paradigm shift) in linguistics over the past 70 years. The unique thing about linguistics is that they've been compressed into the lifetime of the same central figure.

Also, I'll mention that there is a lot of empirical knowledge about natural language syntax in generative grammar, even if the details aren't quite right about their theoretical underpinnings. And a lot of them were "low-hanging fruit" waiting for someone to point out how to find them and characterize them. That Chomsky did that is the best argument for his intellectual value - permitting all that "normal science" to happen.

Science doesn't guarantee technological applications, whether or not that science is correct.

Finally, just saying: use of a word like "Chomskyism" makes you sound like one of those creationists who call evolution "Darwinism".

So we can't use the word "creationism" without validating their use of "Darwinism"?

This is wholly fallacious ... you may disagree with what you interpret as the implications of attaching "ism" to Chomsky, but in no way does that equate it to attaching "ism" to Darwin -- each of these attachments and their validity are independent.

BTW, even what you see as pejorative in such an attachment is a cultural interpretation ... e.g., it is far more common among British evolutionary biologists than American to view "Darwinism" neutrally -- Dawkins for one uses the term and describes himself as a Darwinist. See, e.g., https://www.edge.org/conversation/richard_dawkins-chapter-3-...

The irony is that while (1) is currently dying a terrible death, (2) is on its peak influence of culture and politics

Officially structuralism had a crisis around the release of Syntactic Structures and the morphed into post-structuralism which has been through relentless post[n]-modernization and is whatever it is now.

> Norvig confuses engineering for science. If an engineer wants something that works (and guarantees that it will work) then you use the most effective tool available to achieve that goal. No problem there. But there's absolutely no reason to think that tells you anything scientific about the phenomenon you're modeling.

this distinction is superficial. Scientists use less precise models all the time. Like, you don't solve the schrodinger equation even when you're a scientist doing organic chemistry. You reach for lewis dot models. Science as a discipline is about building better models of the world, not uncovering fundamental truth.

> LLM may be a great technology, but it's probably not a good model for how humans learn and use language.

but from all models, it's the closest, "all models are wrong, but some are useful"

What reason to we have to believe that LLMs are the closest model we have for how humans learn language.

We considered the hypothesis that humans are a general intelligence that figures language out by exposure. The evidence has consistently come back that that is not how humans actually work. A model where humans are hardwired with a lot of knowledge about language, then fill in the gaps based on exposure still appears to be far more accurate.

The only thing LLMs show is that it is possible to learn language de-novo.

> The only thing LLMs show is that it is possible to learn language de-novo.

I agree, but would add, "But not under conditions that humans learn language."

I think it's far too early to say LLMs are not how humans learn language.

Regarding "hard-wired"; it's not if you consider that "hard-wiring" of an individual is the result of millions of years of evolution.

>closest model we have for how humans learn language

Is there some other competing model which can give results as good? If not, the statement seems true in the trivial sense that it's the _only_ model which we've observed to give good results.

People here aren't talking about results, they are talking about the form of the model. Thus the statement you quoted is about how, not about results or outcomes. And it was about how humans learn language, which isn't a result or outcome at all.

> but from all models, it's the closest

Care to back this up? What evidence do we have to suggest that back propagation, self attention, and byte pair encoding are mechanistically similar to human cognition? Similarity of the outputs says nothing about similarity of the machinery underneath. Turboprop engines tell us nothing about how birds fly - doesn't matter that they successfully transport millions of people around the world.

I would say that when it comes to higher level cognition, logic engines are more human like even if they only have a tiny fraction of the practical utility of LLMs.

> Turboprop engines tell us nothing about how birds fly

That is not true.

Turboprop engines and flying birds are different things, but there is a set of common properties.

A blade of a screw propeller functions exactly in the same way as a flapping wing of a bird, except that the wing cannot rotate without limits around the body of the bird, so it must stop, change angles and revert the motion, while the blade can rotate continuously around the axis, functioning on one side like a falling wing and on the other side like a raising wing. The similarities between propeller blades and insect wings are even greater, because insect wings cannot change their shapes so much as those of birds.

For now, our understanding of the networks composed of the neurons of animals is much less advanced than our understanding of wings and propellers, which is why there is a much greater uncertainty about whether there are any similar mechanisms in biological neural networks and in LLMs.

We already know that the most basic mechanisms in ANN and actual neurons are very different. Spiking and neurotransmitters are basically everything in brains and don't exist in ANNs.

Also, AFAIK, insects rely on their (flat) flapping wings to create lift by creating vortices.

"Nothing" may be a little too strong -- physics is physics, after all -- but (small) birds and airplanes have Reynolds numbers about 1000x different and different underlying material science.

Airplanes are engineered (with a very small number of exceptions) to be stable during flight, whereas bird flight is inherently unstable, requiring a lot of specialized cognition so birds can fly.

I would think a lot of cognitive ornithologists (who I really hope exist!) would be interested in finding out about this.

At this point, I'm at the complete end of my knowledge of bird flight!

ok, compare a wheel to a human leg. If you look long enough you will find similarities, but you honestly cannot say that you "understand how humans walk" when all you can do is put a man in a wheelbarrow, move it, and say "look! my device accomplishes the same task!"

> A blade of a screw propeller functions exactly in the same way as a flapping wing of a bird, except that the wing cannot rotate without limits around the body of the bird

The "except" in your argument is doing an incredible amount of work. The rotation of a turboprops generates forward force, wings (whether a bird's or a plane's) generate lift. And plane's wings are called "wings" because they, and not the turboprop, mimic the structure and function of a bird's wings.

Regardless, this whole line of argument is missing the forest for the trees.

attention + residual seems kind of like Chomsky's merge operation.

LLMs behave like they're closest, but only during execution. During training, obviously, they don't resemble the human brain at all. No human reads a million books before they begin to speak.

And I'd be hard pressed to identify many cognitive characteristics of LLMs during execution that resemble human thinking, aside from those that are readily parrotable.

As a model of human cognition, LLMs deep suck.

Humans have billions of years of evolution going for them, that must count…

Humans decidedly take much more data than LLMs. Like orders of magnitude. We happen to feed only distilled data to LLMs though. That's why they appear so efficient.

If you are counting by bytes and including all sensory input, sure. In terms of amount of text, it's not close in the other direction. It's very difficult to make an apples to apples comparison here.

You can obviously count by bytes. They are bytes for the model or for a person. Bytes to bytes is apples to apples.

You can't count by text, because text is not the only thing children learn on. So unlike the bytes comparison the text comparison is totally invalid.

What they said, which is true, is that an LLM's inputs measured in bytes of language is much higher than for humans. Throwing in all the non-language bytes of sensory input received by humans is not valid and thus is not apples to apples. Children learning from watching a dog do zoomies around the room or butterflies in the garden is not learning language.

> We happen to feed only distilled data to LLMs though. That's why they appear so efficient.

That is wild handwaving that really explains nothing.

But then, most of the comments on this page are wild handwaving, pontificating like a blind man describing an elephant. Very little engages with or matches the quality of Norvig's arguments.

> Throwing in all the non-language bytes of sensory input received by humans is not valid and thus is not apples to apples. Children learning from watching a dog do zoomies around the room or butterflies in the garden is not learning language.

Have you ever had kids? Either you never had, you forgot, or you are intentionally deceiving us (and perhaps yourself) to support your point. If you never had kids, I can tell you you have very poor imagination about how they learn language.

The BabyLM challenge has shown that multimodal models don't seem to learn in a more data-efficient way with respect to tokens. Adding an unlimited amount of images doesn't help you learn syntactic islands faster.

Which entry are you referring to?

Not a specific entry, but in four years, there's never been a clear advantage of multimodal models and the results have been extremely lackluckster, even when you allow unbounded images alongside a fixed quantity of text.

> when you allow unbounded images alongside a fixed quantity of text

Any specific entries?

You can look at the CFP for the most recent BabyLM https://arxiv.org/html/2602.20092v2 :

"This year, we have merged the Multimodal and Interaction tracks into the Strict and Strict-small tracks. This is for two primary reasons: Firstly, there were insufficient submissions to warrant standalone tracks. Secondly, none of the submissions in the Interaction track outperformed models submitted to the Strict track, and *none of the submissions from Multimodal outperformed our baselines*. We will still allow participants to use multimodal data or interactive training procedures if they choose, but such data or learning procedures must conform to the rules of the Strict or Strict-small tracks.

Emphasis mine.

> none of the submissions from Multimodal outperformed our baselines

I think that's a pretty weak point to your argument. They are saying that none of the contestants outperformed their baseline multimodal models and that there was very little multimodal submissions in the first place.

Reply on news.ycombinator.com