What Does It Mean to Learn?
newyorker.com
newyorker.com
The real shame is that academic fashion jerked violently in the direction of Derrida and company in the mid-to-late-60s before anyone had time to really dwell with and appreciate the power of Gadamer’s philosophical hermeneutics. His masterpiece was only published in 1960, when he himself was 60! It’s a mature and deep reflection on themes he had been studying for 40 years. Ricoeur grasped its power, of course, but the whole Habermasian-Derridan-Foucauldian critical project flavored Ricoeur’s approach. Richard Rorty gestured in similar directions, with less depth (and certainly far less phenomenological power).
But invest the time to read Gadamer. It’s worth it.
As for the rest, you really do have to read the book. (There’s no Cliffs Notes version.)
I think all understanding and therefore all learning is hermeneutic.
I was persuaded by Hans-Georg Gadamer’s magnum opus, Truth and Method.
I think his theory, which he called “philosophical hermeneutics,” is a skeleton key for understanding our understanding.
Right around the time he published the book, there was a large shift in academic fashion towards critical theory and deconstructionism, of which Gadamer is neither.
The result is that Gadamer and his work took a backseat to critical theory.
We lost something important in that shift.
Others recognized the importance of Gadamer’s work but diluted it by trying to merge it with their pet theories (Ricoeur with critical theory, Rorty with American-style pragmatism).
——————————
That said, Gadamer’s work is dense but not impenetrable. And it is beautiful. And profound. Go forth and read!
[0]https://monoskop.org/images/6/6d/Benjamin_Walter_1936_2008_T...
All can be torn down, and Foucault is the arms-dealer of modernity
But the argument that we should not learn while young because "It's not useful to us then" is a ridiculous premise to argue with. Perhaps I misunderstand it. Then saying that things benefit us far down the road, long after the original moment of learning...
I don't understand this line of thinking at all. Linking this to "educability" as a sort of hidden superpower makes me say "oh, really now?".
Like most scientists, he has to focus his ideas on a track with research directions which expand outwards, excite, and provide new directions and conversationality. He may not be aware of doing this.
Rare is the day when we see a psychologist studying things like how applying a new learning technique to class A effects all the other classes the student takes.
Rare is it we study flow state and motivation of the long term and find meaningful ways at a cohort intra-country level to increase happiness at school or final grades across the board, rather than just one course which the researchers focus on. And when that has been done in Scandinavia, who lead in PISA and happiness as adults and teens, scientists in the US ignore the research by and large. It's sad.
This is what I don't like about education research and theories. They can be worthwhile research directions, but diminished to theorizing, rather than application.
notice that LLMs "context" chat is not learning, it's a temporary effect that gets lost as soon as the chat closes.
is it "learning" in a certain sense? sure, the same way like all indoctrinations are sold as "teaching". the model "learned" to be rapresentative but forgot what 1940 nazi soldier was like...
and fine tuning is not a scalable approach because it has human feeback in the loop. could they fix this error with more fine tuning? yes and they tried abut then the users simply asked "give me a picture a viking warriors" and the problems was again there, you can't fine-tune everything even if we assume the purpose is always noble.
It's simple really they just have the context inserted as part of the input. It's part of the prompt. The prompt is invisible to you as a user.
input:
"You are an LLM with long term memory saved as json objects. When talking to the user you will receive a query from the user and some json memory as context. Add to this memory as you communicate. If the json object exceeds 4000 entries trim 100 entries that seem less important. The users query begins now:
{}
Hi my name is Bernard.
"
"
{"user name": "Bernard"}
Hi Bernard, How are you?
"
If false confidence gives you the ability to learn, the Dunning-Kruger effect isn't so bad after all.
(To paraphrase HP Lovecraft.)
We do know modelling learning will involve fitting a multidimensional curve among a set of data points. There are multitudes of ways to do curve fitting. Our biggest issue, however, is to imitate the way the human brain does it. For this we still don't know how to do learn like a human. But we do know in it's most basic form that learning can be simply fitting a 2 dimensional straight line among a set of two dimensional data points.
At a minimum we need two dimensions. At least one for input and at least another one for output.
A Theory of the Learnable L. G. Valiant 1984
https://web.mit.edu/6.435/www/Valiant84.pdf
Also:
Statistical Learning Theory Vladminir N. Vapnik, 1998
https://www.wiley.com/en-us/Statistical+Learning+Theory-p-97...
And:
Language identification in the limit E. Mark Gold, 1964
https://www.sciencedirect.com/science/article/pii/S001999586...
Anyone who has used StackOverflow (or ChatGPT) to solve a programming problem knows this is completely false. Has the author never heard of “learn by doing”?
A correct answer is not necessarily imply understanding.
Code reviews and incident reports should prove that true.
Additionally not all learning requires some deeper understanding. Sometimes the attempt to mimic is all that there is.
Personally, I think there's a set of outcomes here. To acquire knowledge usefully demands both retention and some ability to apply abstraction/synthesis to the knowledge acquired. Simply being a repositary of knowledge is not in itself usually useful, unless you want to win a competition for the most digits of pi, or ability to recite the Koran from memory, but latent within that ability are other skills which we often decry as "not intelligent" or "simply parroting facts" -the amount of facts you can recall aren't useful in the same way that synthesis over those facts is.
Knowledge, or its synthesis into derivative knowledge, have nothing to do with utility I think ? It's sometimes pointless to acquire knowledge, and useful to act upon the world. So I don't really understand what you meant by "useful" yourself there.
What is the goal to reach ?
There's exterior applicability to knowledge: feed a man a fish vs teach a man to fish.
I see both as "utility" although I suppose achieving nirvana is seen as utility in other people's domains. It's subjective.
Isn't' this IQ? "Educable” seems like the latest iteration of 'multiple intelligences' or the 'street smarts' vs. 'book smarts' distinction.
Not sure what you mean by this - it's certainly not something a single-day IQ test could possibly measure! The primary reason IQ is a discredited measure of intelligence is that people are perfectly able to learn how to perform better on IQ tests - any supposed influence of "intelligence" on IQ scores is hopelessly confounded with how much you've practiced similar tests / trivial logic puzzles / etc.
This stuff about "attempts to broaden the definition of intelligence to something that is more inclusive" is backwards. The whole problem is that nobody has managed to scientifically define (let alone measure) intelligence in any vertebrate species. IQ is dangerously misleading precisely because it is so narrow: its precision makes claims about IQ seem quantitatively rigorous when they are qualitatively meaningless.
Most people, including "intelligent" people, have no clue how learning works. Learning scientists actually study student perceptions of their learning and learners are terrible at selecting appropriate learning techniques. Even when forced to use effective learning techniques, they rate the effective strategies (as determined by their outcomes) as least effective.
We need to start teaching people how learning works to begin with.
Sidenote: it doesn't seem to me that the writer understands AI, its moving frontier, and its likely near-term abilities.
Law students write papers, ER students to CPR on dummies, and when I learn any new concept, framework or language, I build something with it. Don't toy and learning projects count?
No matter how accurate the simulation, the real world is the true test. One could easily see a scenario where a robot is trained to handle a rare event and may or may not ever use that training.
> The "Empty Cup" or "The Empty Vessel" parable: 'A cup that is already full, whether with knowledge, opinions, or experiences, cannot be filled again. It is only the empty cup that can truly learn and absorb new information.'
He makes various arguments in the book that I disagree with, two of which I've put below. On the whole I think it is directionally correct though, and worth reading.
The first quibble I have is about humanity's most characteristic trait. In the book, he writes: "The mark of humanity is that a single individual can acquire the knowledge created by so many other individuals. It is this ability to absorb theories at scale, rather than the ability to contribute to their creation, that I identify as humanity's most characteristic trait".
I don't think that ability to acquire knowledge from other people is our most most characteristic trait. Creativity is. Learning is a form of knowledge-creation, and it is a creative process. We don't passively "absorb" theories when we learn from someone else. Instead, we actively look for and attempt to resolve problems between our existing ideas and the new ideas to create something new.
Another thing I disagree with is when he touches on AGI. He makes the argument that "we should not be fearful of a technological singularity that would make us powerless against AI systems". This is because it will "asymptote, at least qualitatively, to the human capability of educability and no more".
This is reminiscent of David Deutsch's argument that people are universal explainers, and AGI will also be a universal explainer; there is nothing beyond such universality, so they will not fundamentally be different from us (at least, there is nothing that they could do that we couldn't in principle understand ourselves).
I think this is true, but it misses something. It doesn't address the point that there is a meaningful difference between a person thinking at 1x speed (biological human speed) and a person thinking at e.g. 100000x speed (AGI running on fast hardware). You can be outsmarted by something that wants to outsmart you, even if you both possess fully universal educability/creativity, if it can generate orders of magnitude more ideas than you can per unit time. Whether we should be fearful or not about this is unclear, but I do think it is an important consideration.
His overall message though, is good and worth pondering: "Educability implies that humans, whatever our genetic differences at birth, have a unique capability to transcend these differences through the knowledge, skills, and culture we acquire after birth. We are born equal because any differences we have are subject to enormous subsequent changes through individual life experience, education, and effort. This capacity for change, growth, and improvement is the great equalizer. It is possible for billions of people to continuously diverge in skills, beliefs, and knowledge, all becoming self-evidently different from each other. This characteristic of our humanity, which accounts for our civilization, also makes us equal."
It is trivial to prove otherwise - AlphaZero move 37. After 4,000 years of gameplay (yes, it is that old!) we still didn't get this level of insight in its strategy.
The core ability of humans might not be learning but search. Creativity is just an aspect of search. AlphaZero was both searching and learning. It's what we do as well, we search and learn. Science advances by (re)search. Art searches for meaningful expression. Even attention is search. Even walking is - where will I place my next foot?
Why is search a better concept that creativity? Because it specifies both the search space and the goal. Creativity, intelligence, understanding and consciousness - all of them - specify only the subjective part, omitting the external, objective part. Search covers both, it is better defined, even scientifically studied.
Are you saying that AlphaZero contains knowledge that we can't understand, even in principle? It is somehow beyond science, beyond all explanation?
> Why is search a better concept that creativity?
Search suggests a fixed set of options, whereas what is crucial is creating new ones.
I very much agree.
For instance, if you guess 'all swans are white', you don't ever get "statistical confirmation" that your guess was right. When you eventually see a black swan, you find out you were wrong. Then it's time to come up with a new theory.
These cover many creative activities.
But it's true that some search spaces are less well-defined.
These are philosophical questions that really, despite our best efforts, have never transitioned to a true science, and philosophy has been working on it for thousands of years. We've been hamstrung by the fact that as far as we knew, we were the only intelligent beings in the universe, so it's extremely difficult take any aspect of "the way we think" and separate it, by finding some system that thinks in some ways like us, but in other ways doesn't. It's really only been since we've had large neural networks that anything has approached the way we think in _any_ aspect, so this is probably a once-in-history opportunity to formalize and systematize all of this.
The particular problem here is complexity management. The 'problem' with thinking is it is a system, very possibly one of the largest and most complex systems we know about. Thinking as we know it in DNA based life is something that's at least 500 million years old and developed a few bits at a time. There is no unwinding the different components of the thinking from each other. Motor skills, reactions, learning, etc are all compressed and mixed together in the same code.
So, going from top down isn't working. But going the other way isn't working either. The computational complexity of a system that gets anywhere close to a biological thinking system requires quadrillions of calculations.
Chimps (and some other animals) can be very creative: https://www.youtube.com/watch?v=fPz6uvIbWZE
Leslie Valiant, an eminent computer scientist who teaches at Harvard, sees this as a strength. He calls our ability to learn over the long term “educability,” and in his new book, “The Importance of Being Educable,” he argues that it’s key to our success.
People really hate someone pointing out that they have been manipulated by clickbait and cheap marketing though.