Hacker News Reader

The conclusion in Apple's paper rings true – LLMs don't reason, they only match very adeptly the knowledge of their training data. The less commonly a given wrinkle in a prompt appears in their training, the less likely they can accurately handle it.

Will it ever be possible for an LLM to solve a truly novel problem, that doesn't appear in their training data, using deductive reasoning? It doesn't seem like it.

View on HN

I believe that they can already solve novel problems. I can’t find a link but I’ve seen something where ai-skeptical researchers were surprised by chatgpt’s ability to solve certain problems that absolutely wouldn’t be in its dataset.

But of course LLMs can’t handle basic math

I agree about LLM’s alone. But it’s difficult to make confident statements about what machine learning researchers might come up with when combining LLM’s with other approaches.

Reply on news.ycombinator.com