Hacker News Reader

Comment by grumpyprole | Hacker News Reader

I don't understand why anyone is seriously expecting "GPT" to write code? Writing code is really analogous to proving theorems (the Curry-Howard correspondence). IIUC that is not what the GPT "language model" was ever designed for.

View on HN

I don't think anyone expected LLMs to do this well on code problems. Just a few years ago they were struggling to get the syntax right. But since it does seem to work in many cases, it's only reasonable to explore its limits. But in the end, automated math & programming will most likely not be done using LLMs, but with purpose-built systems. The major players are of course already working on this, including OpenAI [1] and DeepMind [2].

[1] https://openai.com/research/formal-math (feb '22)

[2] https://www.deepmind.com/blog/competitive-programming-with-a... (feb '22)

I ask this earnestly, but have you tried it? It most certainly can write a pretty huge variety of code. Yes, it does often have errors, and it struggles with novel or complex problems (as the article points out), but for well known languages and frameworks it is still insanely impressive.

I tried it yes, it did not go well. First it gave me a snippet that looked correct but was not what I was asking for (I guess similar to a search engine). When I tried to point out why the proposed solution did not solve the problem, it could not modify it correctly. It kept oscillating between two incorrect solutions.

Reply on news.ycombinator.com