It can take months for a new developer to understand a legacy code base but an LLM with a big enough context window would be instantly productive.
Once you have the detailed specifications the coding is relatively easy. Some of that can be partially automated using AI CASE tools, but that only gives a marginal improvement to overall project productivity.
One of my first one afternoon (for prompt refinement and trying it out) toy GPT-3.5 “applications”—really just a prompt—was having it interview for reauirements and draft and progressively refine a requirements doc in a specified (by reference to an author in popular Agile literature, not by specific templates, so just relying on the model’s general training) format. Its pretty much what convinced me of the broad utility of the technology.
I absolutely think that with a bit of effort, GPL-4-128K, a custom RAG framework and/or the Assistants API, you could build something that handles a lot of this. I’m not really bullish near term on LLMs as full replacements, but I can see it as a big force multiplier.
So lots of tedious repetitive manipulation of language to extract facts and transform them into a systematic format? My prediction: LLMs are going to eat that for breakfast.
(Honestly it is a great business idea for someone with the right contacts. Lots of big customers with deep pockets and large private datasets to train against that might now have a solution to a problem which was previously intractable in scale.)
one advantage the government would have is training/fine-tuning on a hundred million lines of domain specific cobol.
It's also wildly bad to plan to train and fine tune on code that you know has bugs. Already we have Copilot generating code with trivial vulnerabilities because that's what it's trained on.
Honestly I think editing code will be easier than creating wholly new applications to precise spec.