Don't get me wrong, it has emergent properties (more than you would expect from a fancy autocomplete), but factual output was never GPT-4's nor any other LLM's design goal.
I do agree that others who don’t understand what it’s doing may not even review the code output, and I predict that this will cause at least one death in the next ten years.
I haven't tried gpt-4 yet though, maybe its a vast improvement, I'm just not using it enough to justify the costs
In fact, I bet if you did an experiment comparing people reading a textbook about a somewhat-familiar language, vs. just asking GPT to do things and noticing the bugs, the latter group would reach proficiency faster.
Tell that to 50% of HN who think it's going to replace Google. I think they should be a red disclaimer that answers are often incorrect and shouldn't be trusted in any way.
That being said, I think it's a useful tool for all queries that don't expect a factually correct answer, or can be easily checked. For instance, summary, rewriting, simple code synthesis, and possibly many other niche applications.
To be fair, that's what pretty much every person does. The bar does seem pretty high if we need more than that (especially if not specifically trained on a topic). It's not a universally perfect expert servant, but I've been exploring the code generation of GPT4 in detail (i.e. via the 'cataclysm' module I just posted about). In 1 minute it can write functions as good as the average developer intern most of the time.
We're keeping score in a weird way if we're responding quickly with it needing to "code without subtle but important errors". Because that's the majority of human developers, too. I've been writing code for 30 years, and if you put a gun to my head, I would still have subtle but important flaws in every first typing of any complex generated code.
I'm not saying you're bashing it, by the way, I get your point, but I do worry a bit when the first response is citing that the SOTA models get things wrong in 0-shot situations without full context. That's describing all of us.
If a hammer was only as dense as bone, would it be a good hammer?
Yes, it is possible that such a hammer would still be very useful, if that was the only available option.
Imperfect hammers can still be very very useful, when compared to having no hammers.
GPT-4 is a fantastic collaboration tool for senior developers, who know what they want in detail and can review, verify and apply the output it generates.
Just yesterday I needed to write some detailed bash scripts. I'm no Linux guru but I know what I want and that was enough - in minutes I had a solid script that did everything I needed and wanted, something that would have taken much longer to hunt down through Google. And then I asked it about SQL, C#, AWS, Terraform, Rust and on and on and everything was high quality.
The only way I could have gained similar results without ChatGPT would have been to post all my questions to the dev slack channel and engage in hours long discussions with my colleagues.
But the people who are relevant to the matter work to bring that beyond a minimum.
> and if you put a gun to my head
Like we do with bullshitters? ;)