Hacker News Reader

> it is language that crystallises and captures almost all the human intelligence.

This used to be a common position held by the likes of Wittgenstein, but the biological reality is probably different [1]. Language is probably like the shadows in the Allegory of the Cave, a representation of some thing not to be confused for the thing itself.

[1]: https://mcgovern.mit.edu/2019/05/02/ask-the-brain-can-we-thi...

Of course language relates to things, but I am thinking of language in the sense of a constantly growing corpus of text. It is a self replicator, but like a virus it uses the host organism, a human or a LLM to create more of itself. Ideas have their own life-cycle, they don't get born or die with us, they like to travel and evolve fast.

My point was the capabilities we attribute to LLMs actually come from the training data, not the model. Any model would do, from 10B parameters to 500B, with cosine embeds, rotary or AliBi, even RNNs can do it today. But the data - that is precious, the source of all these "emergent" abilities. We are wondering at the mysteries of the transformer, but the mystery was hiding the training set.

Let's look at the problem from the point of view of language as an evolutionary system of ideas. Like scientific papers and citations, that kind of stuff, leading to AI and human progress. That's where our progress will come from - the evolution of ideas.

> Of course language relates to things, but I am thinking of language in the sense of a constantly growing corpus of text. It is a self replicator, but like a virus it uses the host organism, a human or a LLM to create more of itself. Ideas have their own life-cycle, they don't get born or die with us, they like to travel and evolve fast.

You're talking about "memes", in the original meaning: https://en.wikipedia.org/wiki/Meme

cf https://i.kym-cdn.com/photos/images/original/000/573/606/d0a...

I think it is more that language is a convenient way to capture high quality human intellectual capabilities to train an AI easily. LLMs have highly curated training text to keep that quality high.

In reality we think not just in words but in pictures, sounds, abstract unsymbolized concepts. We also just do with our bodies and just watch with our hearts.

Image models capture some of that. We don’t have a way of capturing our unsymbolized thinking or doing yet. Expect lots of video data to be used to train AI in human body movements.

Unsymbolized, and worse subconscious, thinking will be harder. My assumption is we will have AIs which can learn directly from the world rather than via curated human culture by that point, so won’t need it.

Except highly curated. Not that highly.

My personal take is that "inside" much of language (especially written language) is a kernel of predicate logic. Most pieces of text end up taking a propositional form, eventually. So if you train the right kind of learning machine on an absolutely huge pile of text, it will eventually "learn" some amount of logical constructions, and start producing things that at least look like logical reasoning, in response to logical queries.

Same with training it on programming languages, which have a similar underlying form.

What I've found though is that this "reasoning" starts to get exposed as shallow fairly quickly. It can only "backtrack" through logical relationships a couple levels deep. E.g. I give GPT4 a Rust program with a borrow checker problem and ask it to "fix" it and watch it flail around in circles for 3 or 4 tries before just producing complete nonsense because it gets lost.

It looks like human intelligence like you say, but it just falls over after a few short steps.

Yes, most humans will do the same thing. Except unlike an LLM, humans are able to sense when they get lost and attempt to sometimes retrace or correct their steps. But I think there's something intrinsically missing in LLMs that prevents them from doing that -- they don't "know" why they arrived at some step, and don't seem to be able to retrace and fix. Ask them to fix something they get wrong, and they start to degrade in quality quite quickly, I find. Self-awareness/consciousness may be the kernel of that missing piece.

I think maybe these systems might be more useful once they're tied to an actual rigorous logical deduction system, and "taught" how to use them. Let the LLM "read" the prompt, recognize that it needs to do some logical constraint solving or deduction, etc., create the right terms or horne clauses or whatever in that subsystem, query it, and then let the LLM in turn produce the textual response.

> I think the LLM revolution tell us something else - it is language that crystallises and captures almost all the human intelligence. By training a random initialised transformer on language you get chatGPT. Doing the same to a baby leads to a modern human. It was not the brain or the transformer that are intelligent, it is language running on them that is. Language is a self replicator, but it needed humans, now it works on LLMs too.

This sounds like it's leaning or leading toward Julian Jaynes bicameral mind book or ideas?

What did you have in mind for new features of these created languages? (cf Delaney, Babel 17)