Comment by danaris | Hacker News Reader

Comment by danaris | Hacker News Reader

Parent Full thread

> This brings us to what seems like a contradiction. LLMs are bad at playing games. Yet at the same time, they’re improving rapidly at coding, a skill set that can be used to create a game. How do these facts fit together?

> Togelius: It’s super weird.

...No, it's really not.

They're language models. Code is a language. "Playing a game well" is not. One can, hypothetically, encode game inputs in such a way that it seems kinda-sorta like a language, but it has none of the same kinds of structures that languages—both human and programming—do.

The only way one can think this is strange is if one thinks of LLMs' ability to code rudimentary games as being due to a deeper understanding of games, rather than due to game code being well-represented in their training data.

Yet LLMs can play chess and have a "mental" representation of the chessboard.

If LLMs get better but do not progress at playing games when not specifically trained on it it seems to point to a generalisation failure, a limitation that would prevent LLMs to ever achieve AGI, I do not know if that is weird but it seems that for now nobody really knows if they can achieve AGI or not. Perhaps some emergent behavior will arise after more scaling.

To me it's only totally unsurprising if you are 100% certain that LLMs will never reach AGI (like LeCun thinks for example).

Chess games are in their training set, other games are not.

Chess is representable entirely in text as well, and generally speaking the LLM concept of "picking the next best token" fits pretty well for "picking the next best move" where a move is a text token

That representation is also old, incredibly well documented, and used to describe how to reason about chess. There are of course text guides to other games in training data but they rely upon depictions of what’s happening that aren’t purely text so the game harness is always going to have to make novel decisions about represent the game as text.

Yea it’s wild watching so many smart people convince themselves that LLMs are general purpose AIs. Don’t get me wrong they are incredibly powerful tools. However being surprised that text models cannot play video games particularly well is like being surprised weather models cannot.

Reply on news.ycombinator.com