Hacker News Reader

So is the expectation for it to suggest obvious solutions that the majority of people already know?

I am fine with this, but let's be clear about what we're expecting

All it can do is predict the next token. So, yes.

If it's still 2020, then yes. In 2025 post-training like RLHF made it that these models do not just predict the next token, the reward function is a lot more involved than that.

Instruct models like ChatGPT are still token predictors. Instruction following is an emergent behavior from fine-tuning and reward modeling layered on top of the same core mechanism: autoregressive next-token prediction.

> So is the expectation for it to suggest obvious solutions that the majority of people already know?

Certainly a majority of people don't know this. What we're really asking is whether an LLM is expected to more than (or as much as) the average domain expert.

Reply on news.ycombinator.com