10,610 karma · joined February 21, 2018
I was previously using GLM-5.3 as the orchestrator, after switching to DS anecdotally there was an unnacceptable quality loss, mostly around not taking all the relevant context into account when making decisions, pulling new design out of thin air without discussion too often, and being way too wordy and rambly in documentation despite prompting to avoid it. There's a lot of docs, rulings, core concepts, design philosophy to uphold and DS was just not cutting it.
However, it's perfectly capable of being the sole agent for all of my well specced implementation tasks. I've gone back to GLM as the orchestrator.
You could give me the most well crafted, comprehensive document ever and I would still misread bits of it, or miss some crucial bit of information that contextualises the rest or whatever.
With LLMs I can ask endless dumb questions in a way that just would not be practical otherwise. The quality of the responses doesn't even have to be particularly good for it to be extremely useful - it's basically a turbo charged rubber duck for learning new concepts.
Perhaps some new markets could be entered that weren't feasible before?
They still make up some whole product that presumably does something as a whole that you actually care about, and the complexity inherent to that doesn't go away with either approach.
If you're using a customised harness you should make sure you don't have something that's e.g. changing your system prompt on some requests or rewriting history - it can be tempting to do stuff like strip old thinking tokens or compact tool call results to reduce context size but it's a trap - you want to never change history because of how cheap cache is, even more so with deepseek because their cache hit pricing is so low compared to most other models.
"If the price is high enough" is of course technically true, but the scale of what high means in this context is important.
I realised how use case dependent harness behaviour is when I tried to use my customised-for-a-side-project pi config at work and realised I needed to tweak it significantly to be useful - I would not be surprised if tools like Claude Code needing to be all things for all people is hurting their peak usefulness.
When I say "everything" I mean the frontend, backend, database, seeded data, optimizing runtime etc, I'm not talking about loc coverage. I would not recommend using llms to generate the tests themselves without heavy guidance and review, and you're right that 100% coverage is often a counterproductive goal if the code under test is anything less than pure business logic.