Hallucinations
Context limits
Lack of test coverage and testing-based workflow
Lack of actual docs
Lack of a spec
Great README; cool emoji
Hallucinations
Context limits
Lack of test coverage and testing-based workflow
Lack of actual docs
Lack of a spec
Great README; cool emoji
You have to tell it to validate its own work by adding to, refactoring, and running the tests before it replies.
Most junior developers do care and would never dump partial solutions on a prompter as though they're sufficient like LLMs.
Every time I remember to get `make test-coverage` working and have myself or the LLM focus on lines that aren't covered by tests.
Junior or Senior, an employee wouldn't turn in such incomplete, not compiling assignments that % of the time; even given inadequate prompts as specifications.
A human software developer doesn't code in the void, he interacts with others.
The same when you have an AI coder, you interact with it. It's not fire and forget.
Lack of actual docs
Lack of a spec
Well, not my LLMs at leastWhat does the (Copilot) /tests command do, compared to a prompt like "Generate tests for #symbolname, run them, and modify the FUT function under test and run the tests in a loop until the tests pass"?
Documentation is probably key to the Django web framework's success, for example.
Resources useful for learning to write great docs: https://news.ycombinator.com/item?id=23945815
"Ask HN: Tools to generate coverage of user documentation for code" https://news.ycombinator.com/item?id=30758645
You're better off feeding them a few files to work with, in isolation, if you can.