1,171 karma · joined November 26, 2016
Co-founder: https://kolena.com
Atlas of Space: https://atlasof.space
Live stream validation results as they come in
The body doesn't give much other than the high-level motivation from the person who filed the ticket. In order to implement this, you need to have a lot of context, some of which can be discovered by grepping through the code base and some of which can't:- What is the validation system and how does it work today?
- What sort of UX do we want? What are the specific deficiencies in the current UX that we're trying to fix?
- What prior art exists on the backend and frontend, and how much of that can/should be reused?
- Are there any scaling or load considerations that need to be accounted for?
I'll probably implement this as 2-3 PRs in a chain touching different parts of the codebase. GPT via Codex will write 80% of the code, and I'll cover the last 20% of polish. Throughout the process I'll prompt it in the right direction when it runs up against questions it can't answer, and check its assumptions about the right way to push this out. I'll make sure that the tests cover what we need them to and that the resultant UX feels good. I'll own the responsibility for covering load considerations and be on the line if anything falls over.
Does it look like software engineering from 3 years ago? Absolutely not. But it's software engineering all the same even if I'm not writing most of the code anymore.
- Proliferation of utils/helpers when there are already ones defined in the codebase. Particularly a problem for larger codebases
- Tests with bad mocks and bail-outs due to missing things in the agent's runtime environment ("I see that X isn't available, let me just stub around that...")
- Overly defensive off-happy-path handling, returning null or the semantic "empty" response when the correct behavior is to throw an exception that will be properly handled somewhere up the call chain
- Locally optimal design choices with very little "thought" given to ownership or separation of concerns
All of these can pretty quickly turn into a maintainability problem if you aren't keeping a close eye on things. But broadly I agree that line-per-line frontier LLM code is generally better than what humans write and miles better than what a stressed-out human developer with a short deadline usually produces.
The effort asymmetry is what's rude. The maintainers take their project very seriously (as they should) and are generous enough with their time to invite contribution from outsiders. Showing up and dropping 13k lines of code, posting comments copy+pasted from a chat window, and insisting that your contribution is trustworthy not because you thought it through but because you fed it through a few LLMs shows that you don't respect the maintainers' time. In other words: you are being rude. They would have to put in more upfront effort to review your contribution than you put in to create it! Then they have to maintain it in perpetuity.
Software engineering AI API dependence seems to have already screamed past the tobacco mark but we’re still a long ways away from oil. Though I would bet that we hit it sometime in the next few decades once the bulk of the industry has never written code in any serious capacity.