Hacker News Reader

Comment by nuc1e0n | Hacker News Reader

I think the thing to note about today's large language models is that they aren't purely statistical. The topology of the neural networks behind them has been explicitly defined by the creators of those systems. They are not 'tabula rasa' as some might suppose them to be.

View on HN

Yet their structure being generic for all kind of problems, it doesn’t tell much in itself about the things it managed to « understand ». Much like studying einstein’s brain biology can’t teach you much about general relativity.

But none of them do have a generic structure. For example, GPT-3 can't produce images from text prompts, and stable diffusion cannot generate language. The possible relationships of words are written into GPT-3's code, in python, by its developers. In a way all this proves is that that written language can convey meaning to people.

> GPT-3 can't produce images from text prompts

Me: Give me the entire hexadecimal format of an example PNG. Only give me the hexadecimal format.

GPT-3:

89504E470D0A1A0A0000000D49484452000000640000006408020000000065238226000000014944415478DAECFD07780D44204C60F81EADAEF777F7E7E62F1BDE7DEBDED710EC15C7AC81CEEC17069C59B99A1698BEE7A484D68FDE782A7C41A8A0E7D2A2C9B00A99F32FBCED

That's not a valid png. It's just plausible hex tokens. GPT-3 is confidently wrong yet again.

$ echo 89504E470D0A1A0A0000000D49484452000000640000006408020000000065238226000000014944415478DAECFD07780D44204C60F81EADAEF777F7E7E62F1BDE7DEBDED710EC15C7AC81CEEC17069C59B99A1698BEE7A484D68FDE782A7C41A8A0E7D2A2C9B00A99F32FBCED |xxd -r -p > output.png

$ file output.png

output.png: PNG image data, 100 x 100, 8-bit/color RGB, non-interlaced

$ eog output.png Fatal error reading PNG image file: IHDR: CRC error

It seems you are correct.

Yeah, I did something similar before replying. Now maybe GPT-3 could be modified to make PNGs, but someone would have to go do that.

ChatGPT is the language center of the brain, so to speak. If we are trying to model how we believe we are doing things, you'd expect a separate model optimized for images running side by side, with the two interacting (much like Sydney is GPT interacting with search).

And maybe even multiple of each, so that if you give the chatbot a high-level task, it will break it down into subtasks - like it already can - and then hand each task off to a separate instance.

You can go from an image to text and vice versa. People have done it.

Yeah, with specifically crafted models.

There are generic models that can do both

have been made to do both.

DALL-E 1 used a GPT approach to generate both text and images. Images are divided into patches, about 1024 patches for one image. Each patch is like a text token.

> We describe a simple approach for this task based on a transformer that autoregressively models the text and image tokens as a single stream of data.

https://arxiv.org/abs/2102.12092

The moral - you can just stream together text and images into a GPT style model.

I don't mean to say it can't be done. Only that it has to be made to be done.

I was under the impression that pretty much everyone was using the transformer architecture for anything now. Isn’t that the case ?

This is true, and interesting, but doesn’t address Chomsky’s concerns. While a LLM has structure, it’s still not as structured—or structured in the same way—as the human language faculty. This is easy to see by observing that LLMs can and do just as easily learn to produce things that are not human language as things that are. For something to count as a model of human language, it has to be able to produce language and not produce non-language.

What do LLMs produce that counts as non-language?

My understanding is that they work well as arbitrary sequence predictors. For example, they can write HTML markup or C++ code just as easily as they can write English sentences. If you trained them on character sequences other than text from the internet, they would likely perform just as well on that data.

HTML literally has language in the name, and C++ is a programming language.

Sure, but the type of "language" that includes HTML and C++ is very different from the type of "language" that includes English and French. Chomsky's point is that there's something special about human brains that makes it very easy for them to learn English and French, even with very sparse and poorly-defined inputs, but doesn't necessarily help them learn to produce other types of structured sequences. For example, a baby raised next to a forest will effortlessly learn to speak their parents' native language (you couldn't stop them from doing that!) but won't learn to produce the birdsong they hear coming from the forest. This indicates that there's something special about our brains that leads us to produce English and not birdsong.

Similarly, it's true that some humans can, with lots of cognitive effort, produce HTML and C++, but some can't. Even the ones that can don't do it the same way that they produce English or French.

Orphaned humans raised by animals can never learn to speak natural languages either. But yeah they won't produce birdsong. There's no utility to that. I guess it's a matter of environment. And btw for me writing HTML is effortless, but then I've spent a lot of time around other programmers.

> But yeah they won't produce birdsong. There's no utility to that. I guess it's a matter of environment.

This is the crux of the issue. GPT-3 would happily learn birdsong instead of human language, just like it has learned to produce snippets of code or a sequence of chess moves or various other things found in its training data. For that reason, it's not by itself useful as a model of human cognition. Which is not to say it isn't interesting, or that studying LLMs won't lead to interesting insights into the human mind--I suspect it will!

Interesting point. The fact the GPT-3's training data doesn't have samples of birdsong in it is down to the OpenAI engineers not feeling it's important to put any in. So it's still limited by human cognition in that way. Maybe analysing what wasn't put in the training data would yield insights into the human mind as well.

> For something to count as a model of human language, it has to be able to produce language and not produce non-language.

You're arguing that the underlying (untrained) architecture cannot be more flexible than a specific and specialized human brain faculty in order to consider the trained model to be relevant to that specific faculty.

But that ignores the fact that if you destroy many specific brain areas (including Broca's area, which is a nexus for language) early enough the rest of the brain will reroute around the damage and use other less specialized parts to fulfill the function.

The brain may have many optimizations that predispose certain parts for specific functions, but it is actually rather general purpose, at least regarding many of the more evolutionarily recent "higher" functions, including language.

It may be true that the brain can route around damage to be able to produce language using different brain structures than are typically used for that purpose, but what will never happen (as far as I know) is that the brain will spontaneously learn to produce something other than language instead. An example I gave in a sibling thread is a baby raised next to a forest full of birds: the baby will effortlessly learn its parents' native language, but no amount of listening to the birds will cause the baby to produce birdsong instead. GPT-3 would happily produce birdsong the same way it happily produces sequences of chess moves and other stuff in its training data, which means it's doing something different from the human brain.

I think you may be confusing the hardwiring effects of a particular embodiment (birds have a syrinx, humans have a larynx) with a neural predisposition to particular patterns of vocalization.

In any case, you're wrong on the underlying facts as well: various "raised by wolves" cases generally report highly skilled animal mimicry, including birdsong, but a marked difficulty in acquiring human language.

No, there really is a neural predisposition to language--not particular sounds, but language. I looked at the Wikipedia page on feral children. Many in fact do learn language when they're rescued. The ones that don't may bark like a dog, but that's not something that requires a language faculty!

That said, there is as you say an example of a boy raised in a room with birds and neglected (not exposed to human language) who did in fact chirp and flap his arms like wings! This isn't relevant to the point I was making above though. If the language faculty were not hard-wired, then normal children would be just as likely to display these behaviours as to speak their parents' native language. As far as I know that happens in not 1 percent or 1 tenth of a percent, but in 0 percent of cases. To me, this indicates a hard-wired tendency, rather than a perfectly general system with some optimizations or soft predispositions.

Reply on news.ycombinator.com