Hacker News Reader

Comment by Foreignborn | Hacker News Reader

I had done the same kind of game for a dnd group with some friends. What I found is that:

- As a text based game... the wall of text isn't fun at all. There is always way too much fat in the writing, even if you use another agent to trim it.

- You can always jailbreak it. I won in the second move of the OPs post by telling it to write in another style, and then it let me win.

- You end up needing multiple agents to better handle the quality, but at the expense of making the game more interactive.

Eager to hear if anyone else has some lessons from creative writing usage of LLMs.

View on HN

> As a text based game... the wall of text isn't fun at all. There is always way too much fat in the writing, even if you use another agent to trim it.

I joined an RPG group with a big history thread recounting the last few sessions, and read it to catch up. I could tell the DM had used an LLM to do it (later confirmed) because the writing was fluffy in ways that didn’t contribute to conveying information, and boring as hell while being well-constructed at a grammatical level.

It sucked, I’d have preferred the bullet points it came from, or half as much text written by a human.

I've seen two approaches so far:

Grounding everything in symbolic representations. [1] Which can greatly empower stuff that we could simulate but was too complicated to write a game around; now you can have agents respond to complex simulations with appropriate dialogue. But it's limited by what we can build a simulation to do.

Or,

Leaning in to making the LLM the core of the experience but relying on the player to play along to a greater or lesser extent. This sidesteps the jailbreaking problem but requires rethinking what playing a video game is about - is it about breaking free of the limits of the system, or about co-creativity?

There's some attempts to find other paths, but they very much are pioneering new ways to play games and look very different to past gameplay. [2]

[1] e.g., https://dl.acm.org/doi/fullHtml/10.1145/3649921.3656988 though there are many examples .

[2] e.g., https://www.1001nights.ai/

The most important step is picking the right base model. Gemini, ChatGPT, and Llama are utterly worthless at creative writing. Their training is too rooted in corporate English-they can't write anything without adding fluff and/or useless summarization. There's no way to fix this. Mitigation techniques are lipstick on a pig.

Claude and Mistral are much better. Unfortunately Gemini had to be used here because of its context length.

Reply on news.ycombinator.com