I'm flabbergasted at this comment.
Are you maintaining a list of all invariants you want QA'd by the LLM after each change? How many tokens does this process take?
Are you maintaining a list of all invariants you want QA'd by the LLM after each change? How many tokens does this process take?
Claude Code (Opus 5.5 xhigh), when it makes changes, tests the relevant endpoints for behavior based on what it touched. You could also just explicitly ask it to as a follow-up prompt. You do need to have instructions on how to set up a test environment or use infra for which dev env testing is well-documented online.
And yes, it's non-deterministic. It's not gonna test everything exhaustively or systematically like a fuzzer. It's basically doing what a manual QA would, at best. It probably doesn't even cover every edge case each diff. But I'm fine with this tradeoff for not having unit tests.
Caveat emptor:
1. These are relatively small-medium programs (~10k LOC) where the frontend and backend code reside in the same repo, and the frontend is strictly a web app, and where 100% of the functionality can be tested in the dev environment (incl the database).
2. I'm the only person working on these repos right now.
3. I read every line written by the LLM and have things I don't like refactored or stashed and rewritten with a modified prompt, mostly based on intuition.
>How many tokens does this process take?
I genuinely believe anyone who exceeds usage on a $20 plan doesn't use Claude Code correctly. Always use a new context wherever possible. Always make surgical changes (or, broad changes with well-defined success criteria). For anything complex, get a plan from the LLM at least so that you know it understood your ask. Ctrl+C and `git stash` are your best friends. Keep everything clean, don't build castles on garbage. In the LLM era ironically you should never put off tech debt because it will just compound.
I take a more "hybrid" approach, myself, but I also have control issues. I am slowly moving towards all-LLM, but I'm not quite there, yet.