84 karma · joined April 20, 2023
Perhaps to frame it better: do humans trade with GOD?
Most code on github is by definition average, which is usually a few levels below the quality of an expert programmer. So it's not surprising the output of code LLMs is poor to average quality. For any larger block of code it outputs, a non-trivial amount of editing is required if a higher bar of quality is wanted.
Nice response from Nvidia though.
And also why I'm working on completely migrating off.
My prediction is AMD is already working on this internally, except more oriented around PyTorch not Hotz's Tinygrad, which I doubt will get much traction.
If you're familiar with other fields of AI, adding more and more layers to ResNet was the hotness for awhile, but the trick stopped working after awhile.
If you really poke at GPT, you begin to realize it's fairly shallow. Human intelligence is like a deep well or pond, where as GPT is a vast but shallow ocean.
Making that ocean deeper is not a trivial problem that we can just throw more compute or data at. We've pretty much tapped out that depth with GPT4 and are going to need better designs.
This could only take half a decade or it could be half a century. Plenty of enterprises stagnate for decades.
Wrong. Full self driving is still not here despite access to a huge amount of high quality data.
This guys code might not be as readable as he thinks. I do not want to read through your hundreds of 10 line functions because a few people said "long functions bad".
I liked the portion on naming. My God the amount of AbstractFactorySingletonViewModel garbage I see is astounding.