I came to the conclusion that something in the codebase needs a hard boundary, where the team agrees that only human hands touch it. Otherwise the entire repo becomes untrustworthy as far as discovering intent goes.
8,474 karma · joined June 21, 2015
https://matry.design https://github.com/danielvaughn
I came to the conclusion that something in the codebase needs a hard boundary, where the team agrees that only human hands touch it. Otherwise the entire repo becomes untrustworthy as far as discovering intent goes.
You might think, well code is perfect. But code is syntactically perfect, because it has to be. Because compilers can handle very little ambiguity. But that doesn't mean it's a perfect representation of your thoughts. A huge part of language design is for the compiler, not for the author.
And I'm not 100% sure of this, but I'm fairly confident that this approach would be far more token efficient than the way we currently use AI for programming.
This project maintains a mapping of each line of pseudocode to every line of real code it generates. It's also terse, because it's code, and in my initial tests I've found it dramatically easier to read than a big list of longform specs.
Also love the BDD reference - this is in fact an evolution of an earlier approach where I was trying to combine DDD event storming with Gherkin Rules. Very keen observation.
As for why it needs a dedicated UI, there's a few reasons. First, the syntax highlighting as you mentioned is a wickedly difficult problem. I'm super stoked that it even works at all, tbh. Another reason (which I didn't showcase) is that the editor builds source maps from your pseudocode to the real code. So you'll be able to uniquely trace each part of your prompt to the line it generated.
That kind of thing might be able to be implemented in a plugin, but at the moment it's way easier if I just control the entire environment. I'm not opposed to it, though - this could go in lots of different directions and I'm open minded.
But before AI arrived on the scene, source code was a single artifact that directly expressed the intended behavior of a piece of software as it currently exists. After AI, the artifact is still there, but it's no longer the true record of human intent.
I'll be looking into multi-file stuff soon - it's an interesting can of worms to think through.
My guess is that if you simply write `use some_fn from $repo/some/path`, the LLM _should_ be smart enough to infer in most cases. But we'll have to see how reliable that is.
What I'm after is a condensed distillation of human intent using semi-formal symbolic language, which should be vastly easier to read and understand for engineers and teams.
The issue is that with very large or complex codebases, you tend to forget what was AI generated and what was written by a human. And it's also extremely tedious and difficult to read AI generated code. So if you want to _understand_ a complex bit of code, the natural tendency is to ask an agent to summarize it for you. This can work but also has lots of problems.
What you really want is a system that persists both your written intent, and the actual source code. And you want to provide a source map between them, so that you can understand which bits of human pseudocode are responsible for which bits of generated code.
The real value is in persisting your expressed intent.
edit: fixed
I've tried 3 experiments so far, and have given it serious effort. They're all terrible.
The reason they're terrible is that it's a skill issue. I don't have 3D skills. I don't know the lingo, I don't know how to describe my problems, and I don't know when the agent is shooting itself in the foot.
I've tried a few browsers with vertical tabs, and my brain keeps seeing them as part of the page itself. It's a very distracting and uncomfortable experience.