Hacker News Reader

Comment by fredliu | Hacker News Reader

I have small kids, toddlers, who can already speak the language but still developing their "sense of the world" or "theory of mind" if you will. Maybe it's just me, but talking to toddlers often reminds me of interacting with LLMs, where you would have this realization from time to time "oh, they don't get this, need to break down more to explain". Of course LLM has more elaborate language skills due to its exposure to a lot more text (toddlers definitely can't speak like Shakespeare if you ask them, unless, maybe, you are the tiger parents that's been feeding them Romeo and Juliet since 1.), but their ability of "reasoning" and "understanding" seems to be on a similar level. Of course, the other "big" difference, is that you expect toddlers to "learn and grow" to eventually be able to understand and develop meta cognitive abilities, while LLMs, unless you retrain them (maybe with another architecture, or meta architecture), "stay the same".

View on HN

> Maybe it's just me, but talking to toddlers often reminds me of interacting with LLMs

It's not just you. It hit me almost a year ago, when I realized my then 3.5yo daughter has a noticeable context window of about 30 seconds - whenever she went on her random rant/story, anything she didn't repeat within 30 seconds would permanently fall out of the story and never be mentioned again.

It also made me realize why small kids talk so repetitively - what they don't repeat they soon forget, and what they feel like repeating remains, so over the course of couple minutes, their story kind of knots itself in a loop, being mostly made of the thoughts they feel compelled to carry forward.

And, if you change their context, the story unspooling will change.

Yes. And if they're looped enough in their original story, this feels like the spring from a mechanical watch rapidly unwinding.

It's not just true about toddlers but also for adults in particular time frame. Maturity of thought is cultural phenomenon. Descartes used to think animals are automaton while they behaved exactly like humans in almost all aspects in which he could investigate animals and humans during those times and yet he reached illogical conclusion.

That's a great point. Just thinking out loud, if we can time travel back to the cavemen time, and assuming we speak their language, there would still be so much that we couldn't explain or they wont' be able to understand even for the smartest cavemen adults. Unless, of course we spend significant time and effort to "bring them up to speed" with modern education.

In Jayne's 'The Origin of Consciousness in the Breakdown of the Bicameral Mind', there's some interesting investigation into some of our oldest known tales... Beowulf, The Iliad, etc.

In those texts, emotional and mental states are almost always referred to with analogs to physical sensation. 'Anger' is the heating of your head, 'fear' is the thudding of your heart. He claims that at the time, there wasn't a vocabulary that expressed abstract mental states, and so the distinction between the mind and body was not clear-cut. Then, over time, specialized terms to represent those states were invented, passed into common usage, which enabled an ability to introspect that didn't exist before.

(All examples are made up, I read it more than 20 years ago. But it made an impression.)

I don't think it has anything to do with brain development. I think it's entirely related to the development of an individual concept, whenever the structure of ideas that make the concept is too simple.

I would claim that most people use intuition/assumptions rather than internal chain-of-thought, when communicating, meaning they will present that simplified concept without second thought, leading to the same behavior as the toddler. It's actually trivial to find someone that doesn't use assumptions, because they take a moment to respond, using an internal chain-of-thought type consideration to give a careful answer. I would even claim that a fast response is seen as more valuable than a slow one, with a moment of silence for a response being an indication of incompetence. I know I've seen it, where some expert takes a moment to consider/compress, and people get frustrated/second guess them.

Reply on news.ycombinator.com