The AI could do certain things much faster than I would be able to by hand. For instance, in a certain hash table containing structures, it turned out that the deletion algorithm which moves elements couldn't be used because other code retains (reference counted) pointers to the structures, so the addresses of the object have to be stable. AI easily and correctly rewrote the code from "array of struct" to "array of pointer to struct" representation: the declaration of the thing and all the code working on it was correctly adjusted. I can do that myself, but not in two seconds.
E.g. creating a new page in an app.
Feed an LLM the design, the language/framework/component library you're using, and get a page which is 90% of the way there. Some tweaks and you're good.
Far, far quicker than going by hand, and often better quality code than a junior developer.
Now I would never deploy that code without reading it, understanding it, and testing it, but I would always do that anyway. GPT4 is close enough to a good software engineer, that to resist is it to disadvantage your business.
Now if you're coding for pleasure and the fun of creation and creativity, then ditch the LLMs, they take some of that fun away for sure. But most of the time, I'm focused on achieving business outcomes quickly. That's way more productive with a modern LLM
I firmly believe that the worst kind of help is unreliable help. If your help never does its job, then you know you have to do everything yourself. If your help always does its job, you know you can trust it. If your help only sometimes does its job, you get the worst deal of all because you never know what you'll get and have to check every single time.
Unreliable help is still useful if you know that it's unreliable - you learn to keep a critical eye on what it's doing and correct when necessary. Still saves a ton of time, and I'm not guessing that, I'm saying that from my own experience.
Do you feel the same way about a PR from a coworker?
After all, whether a result is correct or not depends on whether it matches the user's desire, so verification criteria must come from that source too.
Sure there are certain types of relatively objective correctness that most of the time will line up with a user's desires, but this kind of verification can never be complete afaict.
The specific way we're training coding assistants for next-token-prediction would also be an incredibly difficult context for humans to produce code.
Suppose you were dropped off in an society of aliens whose perceptual, cultural and cognitive universe is meaningfully different from our own; you don't have a grounding in concepts of what they're trying to _do_ with their programs. You receive a giant dump of reams and reams of source code, in their unfamiliar script, where none of the names initially mean anything to you. In the pile of training material handed to you, you might find some documentation about their programming language, but it's written in their (foreign, weird to you) natural language, and is mixed with everythign else. You never get a teacher who can answer questions, never get access to a IDE/repl/interpreter/debugger/compiler, never get to _run_ a program on different inputs to see its outputs, never get to add a log line to peek at the program's internal state, etc. After a _lot_ of training, you can often predict the next symbol in a program text. But shouldn't we _expect_ you to be "unreliable"? You don't have the ability to run checks against the code you produce! You don't get a warning if you use a variable that doesn't exist! You just produce _tokens_, and get no feedback.
To the degree humans are reliable at coding, it's because we can simulate what program execution will do, with a level of abstraction which we vary in a task dependent way. You can mentally step through every line in a program carefully if you need to. But you can also mentally choose to trust some abstraction and skip steps which you infer cannot be related to some attribute or condition of interest if that abstraction is upheld. The most important parts of your attention are on _what the program does_. This is fully hidden in the next-token-prediction scenario, which is totally focused on _what tokens are used to write the program_.
This difference is even more stark when it comes to driving assistants. Video compilations of Teslas with FSD behaving erratically and most importantly, unpredictably, are all over the place. Experienced Tesla drivers seem to have some limited ability to predict the weaknesses of the FSD package, but the issue is that the driving assistant is so unlike a human. I've seen multiple examples of people saying "well, humans cause car crashes too," but the key difference is that I have to sit behind the wheel and deal with the fact that my driving assistant may or may not suddenly swerve into oncoming traffic. The reasons for it doing so are likely obscure to me, and this is a real problem.
However, there's a lot we can just throw humans at and trust that the thing will get complete and be correct. And even with the checks and balances, we can have the human perform those checks and balances. A human is a pretty autonomous unit on average.
So far, AI can't really say "let me double check that for you" for instance. You ask it a thing, it does a thing, and that's it. If it's wrong, you have to tell it to do the thing again, but differently.
In all the rush to paint these LLMs as "pretty much human", we've instead taken to severely downplaying just how adaptable and clever most sentient beings can be.
In any case, the point is that we have learned techniques to compensate for human fallibility. We will learn techniques to compensate for gen AI fallibility, as well. The objection that AIs can be wrong is far less a barrier to the rise of their utility than is often supposed.
The original argument put forth was that "an inherently unreliable tool cannot gauge its own reliability".
Someone responded that humans fit that description as well.
I said we don't. We can and do verify our own reliability. We can essentially test our assumptions against the real world and fix them.
You then claimed we couldn't do that "self-sufficiently".
I responded that while that is true for some tasks, for a lot of tasks, we can. That an AI can't check itself and won't even try.
And now you're telling me that they can check against each other.
But if you can't trust the originals, asking them if they trust each other is kind of pointless. You're not really doing anything more than adding another layer.
For example: If I put my pants on backwards, I correct that myself. Without the need to check and/or balance against any other person. I am self-correcting to a large degree. The AI would not even know to check its pants until someone told it it was wrong.
The objection isn't that "AIs can be wrong", the objection is that AIs can't really tell the difference between correct and incorrect. So everything has to be checked, often with as much effort as it would take to do the thing in the first place.
Your objections seem to rely on a restricted view that says "we can't do better" but with no evidence. Whereas we have plenty of evidence of massive, continual improvement in the very areas you are holding up as problematic.