I’ve experienced this myself. I’ve had to downgrade my opinion of my own code. LLMs are great at pointing out the edge cases that you might have taken years to find. Honestly feels quite dispiriting but we move on.
Previously my main concern was with LLM's writing worse code. Now my concern is the ability to write better code than I was ever able to write before -- considering every possible potential bug and race condition and error state and what have you -- but at what cost? How much more time do you have to spend researching, designing, making decisions around every possible edge case and state?
Before, you couldn't even attempt to balance quality with speed, because you couldn't see the quality in advance. You had to wait for bug reports to come in. Now, you're confronting quality up-front in a much deeper way. Which allows for much more reliable software, but then you have to decide how much effort is worth it and how much isn't?
a particular statement to make during the slop apocalypse
You can not know what you're doing and use it to write slop.
Or you can set up your project with best practices in every way with full design documents and code that flows from design and full rigor around everything.
I do it the second way. And so yes, I find myself being confronted with so many edge cases that I just never thought about before. And I'd say it about doubles the time it takes me to deliver stuff, and I'm not always sure it's worth it.
In it he pointed out a huge deficiency of most programming languages that still haunts us: they’re wickedly difficult to reason about. So proving important properties about programs is all but infeasible.
So we try to handle it empirically with unit tests, but that has its limitations.
LLMs can kinda sorta do it analytically, but that isn’t completely satisfying, either. The results are probabilistic, and they are pretty bad at inferring intent which can be a source of both type I and type II errors. But still, it would be silly to throw out a layer of Swiss cheese for having holes.
But I sure do wish we could have a plausible version of Backus’s original vision of a language that lets us formally declare (a useful subset of) the specification and then statically verify that the program conforms to it.
Being able to take an old code base up to 100% test coverage alone is incredibly eye opening.