Don't get me wrong, it has emergent properties (more than you would expect from a fancy autocomplete), but factual output was never GPT-4's nor any other LLM's design goal.
I do agree that others who don’t understand what it’s doing may not even review the code output, and I predict that this will cause at least one death in the next ten years.
I haven't tried gpt-4 yet though, maybe its a vast improvement, I'm just not using it enough to justify the costs
In fact, I bet if you did an experiment comparing people reading a textbook about a somewhat-familiar language, vs. just asking GPT to do things and noticing the bugs, the latter group would reach proficiency faster.
To be fair, that's what pretty much every person does. The bar does seem pretty high if we need more than that (especially if not specifically trained on a topic). It's not a universally perfect expert servant, but I've been exploring the code generation of GPT4 in detail (i.e. via the 'cataclysm' module I just posted about). In 1 minute it can write functions as good as the average developer intern most of the time.
We're keeping score in a weird way if we're responding quickly with it needing to "code without subtle but important errors". Because that's the majority of human developers, too. I've been writing code for 30 years, and if you put a gun to my head, I would still have subtle but important flaws in every first typing of any complex generated code.
I'm not saying you're bashing it, by the way, I get your point, but I do worry a bit when the first response is citing that the SOTA models get things wrong in 0-shot situations without full context. That's describing all of us.
If a hammer was only as dense as bone, would it be a good hammer?
Yes, it is possible that such a hammer would still be very useful, if that was the only available option.
Imperfect hammers can still be very very useful, when compared to having no hammers.
GPT-4 is a fantastic collaboration tool for senior developers, who know what they want in detail and can review, verify and apply the output it generates.
Just yesterday I needed to write some detailed bash scripts. I'm no Linux guru but I know what I want and that was enough - in minutes I had a solid script that did everything I needed and wanted, something that would have taken much longer to hunt down through Google. And then I asked it about SQL, C#, AWS, Terraform, Rust and on and on and everything was high quality.
The only way I could have gained similar results without ChatGPT would have been to post all my questions to the dev slack channel and engage in hours long discussions with my colleagues.
But the people who are relevant to the matter work to bring that beyond a minimum.
> and if you put a gun to my head
Like we do with bullshitters? ;)
Tell that to 50% of HN who think it's going to replace Google. I think they should be a red disclaimer that answers are often incorrect and shouldn't be trusted in any way.
That being said, I think it's a useful tool for all queries that don't expect a factually correct answer, or can be easily checked. For instance, summary, rewriting, simple code synthesis, and possibly many other niche applications.
It doesn't help me writing Unreal C++ code, though it's heavily documented online. It makes up functions that are not accessible from the namespace I'm using, cannot supply valid types to parameters and other basic errors like that.
I asked GPT4 to give me specific examples of the meaning of the word "silly" across the past 1000yrs. It gave me the authors and examples I was expecting to read, but the meanings were inaccurate and misleading.
If there's GPT Power I'm missing out on, please tell me where it is. It's probably my lack of skill, that the machine cannot give me an experience that would push me into the pro-GPT cheerleading camp.
But the human will learn, through reasoning and challenges... Those features of which some wonder whether they will emerge are critical pieces.
That depends on the human, what they're trying to learn, and how it's taught to them.
Some people with brain damage, learning disabilities, other cognitive issues, with poor teachers or learning materials, or with too difficult a task will not learn.
It might be useful to view AIs as akin in some ways to brain damaged humans, who might be good at some tasks but deficient at others.
I have used it for many non software things as well, and again very useful.
I think my needs are too specific, and often too nebulous.
Also, just a bit chapped about falling for the hype.
I have a hunch it’s much better suited to get context specific summaries for further good old “RTFM”. And that in itself would be cool enough if people would actually go and “fact check” the outputs which I guess (and telling from comments) is often not the case.
It kind of reminds me a pseudo-visual code builder.
I was writing a Python script with it to put together documentation summaries, and over the course of an hour it offered me 3 different libraries to connect to an API endpoint I was using. It's distracting.
Could you expand on this? I only use the unpaid API, but when I ask for, say, song lyrics, it'll be 100% wrong and then it will give me a second version of the wrong lyrics when I say the first was wrong, and then when I ask how confident it is in the new lyrics it'll say 95%, so whatever the current free API is it's undeservedly confident. I must be misunderstanding what you mean by confidence.
I am comparing it directly to 3.5. Again, I don't have the API access so I cannot say for sure, this was only my experience from the ChatGPT interface and also a third-party API I spent $20 on just to test its limits, but sadly for the time being those limits are hardcore - it just gets stuck on long responses despite there being a token increase.
So, if I really wanted to test it out and make a proper assessment, I'd need access to the API myself so I can do some high-volume token shenanigans.
I've tried 3 separate projects with it (using different platforms from Firebase to Vercel) and it usually gets me 80% there but fails miserably at the nuance portion (its stuck at a CORS policy issue with my last project).
From start to finish I've yet to be able to complete something with it.
Seems like something they might want to include, especially when its a paid service.
I asked it to describe myself recently, by my name and country of residence, and it replied with the correct occupation (founder), and then completely hallucinated a company I have never created. The more you ask it minute tidbits of information, the more it falls over into a bullshitting state.
Nonsense.
I'm not saying that it is not useful, but I've easily found ChatGPT produces far worse than "parroted bullshit", and does so reliably, and multiple times (repeated patterns of mistakes).
Again, I have found ChatGPT4 (but not 3.5) useful and even educational in some cases, but it's usability requires far more skeptical and error-checked approach.
As one example, since I have a top specialist attorney in the household, I started with some broad legal questions about their field, and then asked their opinion. ChatGPT4's answers were impressive to both of us, and especially the solid grammar.
When the questions got more particular about their specialist area of the law, ChatGPT4's answers got shaky very fast. Asked about a particular somewhat obscure technique, it parroted some very nice-sounding bullshit, which got the concepts very backwards (but it was still very confident and grammatical).
When asked to summarize a published paper about a new innovation in this context, ChatGPT4 very grammatically and confidently completely missed every significant point, again in a very backwards way.
Worse yet, when asked to tell us the paper's publication date, author, and author's employer, all written clearly at the top, it confidently got everything massively wrong. Even after being told to look on the line after "Author: ". Even after being told to look after "Author: FName". It eventually settled on the correct date, but even after being told FName, it continuously kept spouting false authors (sometimes including correct FName) and hallucinated middle and surnames and employers. Interestingly enough, most of the hallucinated authors and employers were well known in the region. So, it is literally establishing the regional context and just spitting out plausibly relevant stuff for that region, but not even figuring out how to summarize something that a 6yo kid could easily sort.
It was hilariously bad. And only hilarious because it was a test (if someone actually relied upon it, it'd be very costly).
Similar results for my field. I got ChatGPT4 to produce some genuinely useful information. But it would also happily hallucinate papers, authors, companies, chemicals, and pretty much anything else.
Middle-of-the-road, where ChatGPT4 has ingested insane amounts of data, it probably knows more than we do, and may be useful for some things.
As soon as you go even slightly off the centerline, it becomes VERY unreliable.
If you take it at face value, you are a fool, being fooled.
EDIT: typos, readability
And if it is excluded, why wouldn't it just default to a response resembling "I am not setup to identify and extract that data."?
Also, the fact that it could pick up the provided FName and still hallucinate MidName and Surname and Employer kind of gives away that it was in context.
It is really funny (as long as it isn't a serious situation) how it cannot identify anything, but the context will give it cues to hallucinate