Hacker News Reader

Orphaned humans raised by animals can never learn to speak natural languages either. But yeah they won't produce birdsong. There's no utility to that. I guess it's a matter of environment. And btw for me writing HTML is effortless, but then I've spent a lot of time around other programmers.

View on HN

> But yeah they won't produce birdsong. There's no utility to that. I guess it's a matter of environment.

This is the crux of the issue. GPT-3 would happily learn birdsong instead of human language, just like it has learned to produce snippets of code or a sequence of chess moves or various other things found in its training data. For that reason, it's not by itself useful as a model of human cognition. Which is not to say it isn't interesting, or that studying LLMs won't lead to interesting insights into the human mind--I suspect it will!

Interesting point. The fact the GPT-3's training data doesn't have samples of birdsong in it is down to the OpenAI engineers not feeling it's important to put any in. So it's still limited by human cognition in that way. Maybe analysing what wasn't put in the training data would yield insights into the human mind as well.

Reply on news.ycombinator.com