For most big software projects the bottleneck is not writing code and test boilerplate that runs correctly locally. Which of course LLMs can do reasonably well. Most time is spent on design, on integrations with other internal services, scaling concerns, and reasoning about other things that these models can't really reason about yet.
I'm obvious impressed that these models can generate simple client/server websites and generate code to solve competitive programming problems. But I don't think they've "solved" software engineering; they've only solved "what you boss thinks software engineering is".
That's still bad, though, because your boss makes hiring and salary decisions.
Maybe in the end it's a kind of "great filter" - companies and bosses that can't tell the difference don't survive. But a lot of good programmers can get fired before the clueless bosses die off...
The cause of the inefficiency and "great filter" you're describing is the labor protectionism currently in the U.S.
Theoretically a good programmer has arbitrage via their skills and expertise. If you can get things done people will pay you for it, assuming you're allowed to work (in the U.S., EU, etc.).
What is it really? Just an industry plagued by nepotism?
Which is why bosses are excited but engineers are not, because subpar LLMs will be used to justify firing actual engineers and replace them with fresh college grads tasked to work with a code bot, because the bosses fundamentally think that is enough.
If a new grad + LLM can do task X about as well as an experienced engineer can, then the experienced engineer will have to find other harder problems to solve. Personally I think labor efficiency is good; why waste your labor on easy/solved problems?
The experienced engineer also gets an LLM though
Say experienced engineer Edna was working on a project and the new grad Nelson replaced her because he was cheaper and about as good if both have LLMs, i.e.,
Edna + LLM ≈ Nelson + LLM
even though
Edna >> Nelson.
So Edna needs to find a project to work on where LLMs don't equalize her arbitrage, i.e.,
Edna + LLM >> Nelson + LLM
and Nelson needs one where
Nelson + LLM >> LLM.
Project fails = bad.
If a bunch of projects given to NG + LLM fail, the company will say at some point "this is no good, let's not do that anymore". If they mostly succeed, they'll do it more.
From a middle manager's perspective, where they're funding 10-20 projects, this kind of measurement is feasible and carries at least some signal. So I don't agree it's difficult to measure.
I actually don't relate to this "bosses don't know what software engineering is" thing at all. All my bosses are older-school engineers-turned-management who are more skeptical of this than the young'uns.
You want to increase churn? Are you insane?
I feel like if you poll computer / phone users (ie., the vast majority of the population) one of the most annoying things users hate about modern devices are mandatory updates, like the infamous meme video where someone screams at their window machine updating. Now, you want to make it easier for middle managers to push new unneeded updates so they can pad their resumes, how absurd.
On the other hand, if the models can be Atkinson-like and produce -2000 lines of code [0], that is a day to which I would look forward.
0. https://www.folklore.org/StoryView.py?story=Negative_2000_Li...
Could that be iterated? Can a half baked release ever become whole or does progress just slow down and stretch out exponentially? 1/2 + 1/4 + 1/8 + ...
Will be grateful and relieved when proven wrong.