7 karma · joined November 21, 2022
However, once you want to trigger tasks from Slack, Linear, or GitHub issues or onboard teammates who aren't comfortable wiring up LXC + tmux + agent forwarding, a managed layer is needed.
I think we're at a moment where builders with great setups like yours and products like ours are feeding each other good ideas. The patterns you figure out in your zshrc inform what we productize, and the workflows we ship give you new things to try. It's a virtuous circle. Everyone should use the right-sized solution for their situation.
You are also free to swap/combine these harnesses as you please, which is something Anthropic can't do. For instance claude code implements and codex reviews.
The ralph loop mode also has the concept of a budget per task.
One learning we had is that most of the time CLI > MCP
Defining the plan/acceptance criteria for long running task is the hard part.
We recently added a Ralph loop mode in that spirit. The implementation won't start until the human and agent align on verifiable criteria and a different agent judges if criteria are met at the end of each run.
Overall I think this problem is not yet completely solved and improvement on both the UX and model judgement are needed
However reusing the GitHub workflows out of the box feels really nice
I am not saying that build is always the bad choice, but the tradeoff did not disappear imo
What Anthropic can't offer is to let you use Codex or combine it with Claude Code. That is why we think non ai-labs players have a say in this market.
To your last point, as always there is a buy vs build tradeoff which ultimately comes down to focusing on your core business which we think still remains important in the ai era
That said, there are workarounds, like cloning all repos and enabling LSP (coding CLIs added that feature) or using a dedicated solution for codebase indexing and add a skill/mcp.
Super fast models spamming grep commands are also fun to watch!
Curious to know how you implemented it in house.
This is what enables Twill to self verify its work before opening a PR
For the lightweight sandbox, can you give an example?
Currently we support main coding CLIs, ACP support is not shipped yet.
Also you can fire and forget tasks (my favorite) and don't have to keep your laptop running at night.
On the cost for solo devs, yeah, if you're one person running one agent at a time on your laptop, the sub is probably the better deal today. No argument there. The cloud agent model starts to make sense when you want to fire off multiple tasks in parallel.
I love the idea of emailing agents like we email humans! Thank you for sharing your learnings:
1. Network constraints vary quite a bit from one enterprise customer to another, so right now this is something we handle on a case-by-case basis with them.
2. We came to the same conclusion. For sensitive credentials like LLM API keys, we generate ephemeral keys so the real keys never touch the sandbox.
3. Totally right, we support constrained tasks too (ask mode, automated CI fixes). We've gone back and forth on whether to go vertical-first or stay generic. We're still figuring out where the sweet spot is. The constrained tasks are more reliable today, but the open-ended ones are where teams get the most leverage.