[1]https://system76.com/support/difference-between-pop-ubuntu
490 karma · joined March 9, 2018
[1]https://system76.com/support/difference-between-pop-ubuntu
I never considered “language stability” until your comment, but I think you are right. I’ve had far fewer dependency upgrade pains and a general feeling that bit-rot is slower overall.
Older apps that depends on Node + trendy npm packages + popular frameworks can be a real nightmare to bring up-to-date.
My aim was to voice disagreement with the "code is spec" crowd, whom I think are using a different (and in my opinion tautological / useless / counterproductive) definition of spec. Probably because they are mad that I use trigger words like Vibe and Maxxing, and they assume I can't even read the code I'm shipping. I digress.
In your "time complexity is a downstream requirement" example, which is a great one, I think you would prefer to have well-maintained written documentation of that criterion that lives outside of the procedural code itself, would you not? How much attention that doc gets is a matter of process and preference, but I'm advocating it should get more (spec-first).
The tooling expects that you refer to each requirement by full ID e.g.
# admin.AUTH.11. Start with unmaintainable prompt.
Add auth to the admin route. It must only be accessible to team members. It must not be accessible to team members with the `quarantine` role. Unauthorized users should be redirected to the /login page, with query param `?error=unauthorized`. Requires fresh session, no older than 10 minutes.
2. Put it in a structured .md spec of some kind # Feat: Admin Route
## Auth requirements
* [ ] Accessible to team members.
* [ ] Not be accessible to members with `quarantine` role.
* [ ] Redirects unauthorized users to /login
* [ ] Redirect path includes `?error=unauthorized`
* [ ] Requires fresh session, no older than 10 minutes.
But how do you cross reference the requirements? Give them stable IDs? # Feat: Admin Route
## Requirements
AUTH.1: Accessible to team members.
AUTH.2: Not be accessible to members with `quarantine` role.
AUTH.3: Redirects unauthorized users to /login.
AUTH.3.1: Redirect path includes `?error=unauthorized`.
AUTH.4: Requires fresh session, no older than 10 minutes.
Might as well clean it up and make it machine readable at that point? feature:
name: admin-route
requirements:
AUTH:
1: Accessible to team members.
2: Not be accessible to members with `quarantine` role.
3: Redirects unauthorized users to /login.
3-1: Redirect path includes `?error=unauthorized`.
4: Requires fresh session, no older than 10 minutes.
```To me saying "the code is the spec" is like saying "the business wants it this way because that's how the code is written". Which is obviously backwards.
Does the business mandate we use a cache for this hot path? No, but the business set performance targets, and the cache was a sensible way to satisfy them. See the difference?
I believe that the 'musts' and 'must nots' deserve special attention, and need to be recorded well before I decide on the 'how'. Every team does this differently. I find that writing itemized, functional acceptance criteria is practical way to marry the two domains. I also think the process matters a lot more now, because the temptation to let an agent ship it is increasing and the tedium of maintaining these specs is decreasing.
Disagree on the bit about it "never going to work" though.
Failure-prone stochastic ML systems produce testable, auditable code... just like failure-prone human brains can produce testable, auditable code. And in fact, in both cases, changes to our process can reduce the amount of failures that slip past testing and audit. Or can reap other rewards. Finding the a better process is what I'm interested in right now.
Second is that I'm doing a lot less "seat of my pants prompting" and doing more engineering and ideating, which was a big goal of mine. So I'm feeling less psychotic there too.
And sort of tangentially to that, I think a significant subset of devs actually are willing to just prompt their way to nirvana, day in and day out. I'm not. I think the spec will carry a lot of weight for a long time. Maybe they will get further than I give them credit for? Maybe the whole digital world becomes a single chat box?
> My point is, the spec must live somewhere, even if you don’t write it down. The spec is what you want the software to be. It often exists only in your head or in conversations. You and your team and your business will always care what the spec says, and that’s never going to change. So you’re better off writing it down now! And I think that a plain old list of acceptance criteria is a good place to start. (That’s really all that `feature.yaml` is.)
I went down quite a rabbit hole experimenting with what I call “specsmaxxing” (wrote about it too, see my post history).
My first reaction is that everything I see defined here in DESIGN.md is already codified in my actual themes configs, or component files.
I understand what they want to achieve, as LLMs almost never get design right on the first shot, and very often they ship something that is functionally to-spec but still visually unshippable.
The thing is, you can hand-implement a feature exactly as defined in the most well-prepared design briefs and mocks, and still find something worth changing as soon as you play with the working version. You might not appreciate how fluid layout and styles and themes can be, until you try speccing them out (and later finding yourself updating the spec to chase the product). IMO trying to codify design rigidly and prescriptively in docs is going to slow you down more than it’s worth.
Maybe there is more value for teams trying to align design across dozens or hundreds of products though?
When it comes to acceptance criteria though I am fully on-board. We need to be writing requirements down and tracking them atomically through implementation and QA, and maintain them in long lived feature requirements docs. Yaml is great for this. I built some tools that help me do that in a very productive way already.
The big expensive models are great at planning tasks and reviewing the implementation of a task. They can better spot potential gotchas, performance or security gaps, subtle logic and nuance that cheaper models fail to notice.
The small cheap models are actually great (and fast) at generating decent code if they have the right direction up front.
So I do all the spec writing myself (with some LLM assistance), and I hand it to a Supervisor agent who coordinates between subagents. Plan -> implement -> review -> repeat until the planner says “all done”.
I switch up my models all the time (actively experimenting) but today I was using GPT 5.4 for review and planning, costing me about $0.4-$1 for a good sized task, and Kimi for implementation. Sometimes my spec takes 4-5 review loops and the cost can add up over an 8 hour day. Still cheaper than Claude Max (for now, barely).
Each agent retains a fairly small context window which seems to keep costs down and improves output. Full context can be catastrophic for some models.
As for the spec writing, this is the fun part for me, and I’ve been obsessing over this process, and the process of tracking acceptance criteria and keeping my agents aligned to it. I have a toolkit cooking, you can find in my comment history (aiming to open source it this week).
I sprinkle in some billed API usage to power my task-planner and reviewer subagents (both use GPT 5.4 now).
The ability to switch models is very useful and a great learning experience. GLM, Kimi and their free models surprised me. Not the best, not perfect, but still very productive. I would be a wary shareholder if I owned a stake in the frontier labs… that moat seems to be shrinking fast.
I find the added structure of yaml + requirement ids helps tremendously compared to plain markdown -
I am still a few days away from open sourcing the stack (CLI / API & Server), plan is to gather as much feedback as I can and decide if this is worth maintaining.
1. Writing a spec with clear acceptance criteria.
2. Assigning IDs to my acceptance criteria. Sounds tedious, but actually the idea wasn’t mine, at some point an agent went and did it without me asking. The references proved so useful for guiding my review that I formalized the process (and switched from .md to .yaml to make it easier).
3. Giving my agents a source of truth to share implementation progress so they can plan their own tasks and more effectively review.
Of course, I can’t help myself, I had to formalize it into a spec standard and a toolkit. Gonna open source it all soon, but I really want feedback before I go too far down the rabbit hole:
So far this workflow is the only way I’ve been able to have any real success running parallel agents or assigning longer running tasks that don’t get thrown out.