Hacker News Reader

Comment by andai | Hacker News Reader

> “LLMs are Bad At Games”

> Any time I tried to use an LLM for a game-y thing, it would fall over.

You know it's funny. I did a bit of LLM-assisted game dev this year.

I spent a few weeks working on a simple Pong game, just polishing it, making small changes here and there. Frontier models couldn't change or add a feature without breaking something else. It was still more productive than working "solo" but quite annoying.

Meanwhile, I asked a model to make an entire game -- about 100x more complicated -- and it just one-shotted it perfectly, no issues.

I'm not sure how both of those things can be true at the same time. Something something "in-distribution" ?

View on HN

Yes, that's about right. It is much easier for it to roughly copy other code that it has seen, than to understand the code it has been given enough to make targeted fixes.

When you start from the end, model may make the proper architecture for everything you will need, and for everything it needs to know that implementations is correct.

I think verifiable game engines are slowly starting to appear. After all, nothing can stop you from unit-testing the game like Factorio does ( https://youtu.be/CgMV2dFFdFE?t=48 ) other then your not bothering to think about it. I have made one even, see https://github.com/Dimava/gensokyo-survivors/tree/master

> this year

Models change so fast now that even "this month" becomes abstract. Everyone, please make sure to mention the name of the model.

> Frontier models couldn't change or add a feature without breaking something else.

Have you tried working with Rust (e.g. Bevy) and all of its compile-time guarantees to prevent that? I imagine it must help a lot.

To clarify, there were no compile or run time errors, it just kept breaking the game.

The code might've been garbage in the one shot case. But it doesn't really matter IMHO

Yeah sounds like you didn't polish the new thing.

The hard stuff that comes after "implementing the basics" and released game. The stuff that is hard to even verbalize and takes lots and lots of iteration. Good luck using AI for all that.

That’s 100% it.

I one shotted a math game for my 4 year old complete with lots of juice “screen shake, explosion animations etc…”.

It worked great, but if I tried to change even the slightest thing the result was worse than the initial output. No matter how hard I tried to nail things down with tests and specs, each change looked or felt worse.

Reply on news.ycombinator.com