Hacker News Reader

Comment by throwerofstone | Hacker News Reader

It is quite amusing to read this essay 8 years after it has been published, now that we know about the similarities of how large language models (LLMs) and the human brain function. The average brain does indeed not copy processed information perfectly, as the author demonstrates. It generalizes and creates or strengthens connections between neurons that represents that information. Just like how LLMs increase or decrease their own weights.

View on HN

I was irritated by how strongly the author insists that "nothing is stored in the brain", almost to a point where the author seems to suggest it's stored somewhere else entirely, when after paragraphs of paragraphs the gist of it is that information is not stored directly, like in the case of the dollar bill, not as a bitmap, so to speak. There is still some form of incomplete, lossy encoding of information and knowledge stored in the brain. I don't see how this would invalidate the analogies with computers.

This isn't new or groundbreaking in any way. Cognitive science basically tries to figure out the "algorithms" at work when we store or retrieve information. They're still analogies of course, but that we don't have an image of a dollar bill stored in some synapses hardly isn't news to anyone in the field.

> almost to a point where the author seems to suggest it's stored somewhere else entirely

These "the brain isn't a computer" essays never quite say it but obviously assume the existence of a kind of "soul".

Isn't it simply a PCA/DimRed?

i.e. I am the sum of all trillion of my features, but I am also mostly the sum of a set of a few thousand informative ones combined in linear/nonlinear ways

You could drop a verse or two of Shakespeare from my memory and I'd probably still be recognisable to myself and those around me

The whole idea that memory is stored in a distributed, lossy, and redundant fashion is hardly a new one; I read about the concept of Sparse Distributed Memory in Science News as a youngster more than 3 decades ago, and it in turn was informed by earlier ideas of sparse and spatial-coded memory (e.g. holographic metaphors of recollection).

LLMs provide evidence that you can build systems with these exact properties; no individual perceptron stores a concept, and the encoding is extremely sparse and redundant.

Of course, LLMs don't demonstrate conclusively that the brain works this way, but given that this form of information storage and retrieval works across a real analogous system refutes those that said this would be impossible for the brain to do.

We know no such thing. The human brain behaves nothing like an LLM.

This article stands the test of time, whereas you appear to be subscribing to the fallacy that whatever strange new ideology comes out of silicone valley is the truth.

Do check out neurology some time.

Neural networks are a total misnomer and absolutely butcher any biological insights it may have been loosely based upon.

You seem to not understand LLMs nor the human brain if you think the article stands the test of time. Even the field of neuroscience 8 years ago would have issue with the claims made in the article.

Hey, can you please make your substantive points without resorting to the flamewar style? We want curious, respectful conversation here.

This is especially important when you're advocating for a minority view, because if you post like you did here, you just create an additional reason for people to reject what you're saying.

https://news.ycombinator.com/newsguidelines.html

Reply on news.ycombinator.com