142 karma · joined November 29, 2018
Disclaimer: the blog should be glanced high-level as human, read the below instead. You can point your agents to it tho.
We've been working on agent-swarm (https://github.com/desplega-ai/agent-swarm) as an oss MIT project for a while.
One of the core principles on why we believe in is that when you adopt AI internally in your company, you should be free and not tied to a single provider (e.g. Devin).
Yesterday Devin introduced the Devin Oncall, and we wanted to replicate the feature in the swarm, but with one thing in mind: what if Devin is down?
As the swarm is built on top of a set of building blocks (not only agents), we created a set of workflow, deterministic scripts, and ui so that when a Pagerduty (or any other) comes it will handle it with automatic provider failover.
It's a simple thing really. In the blog post our swarm created explaining how to set it up you can see the high-level wf, and sub-wf.
If for some reason you have a swarm running point it to it. If you have some other system, do it too, as it might give you the high-level arch. to replicate it.
Cheers,
We've been building the swarm for almost a year now, and I'm super excited that our vision to build this oss (MIT) and harness agnostic is more relevant than ever.
I wanted to share the project again, as some of the latest things we did that I believe are relevant:
1. Scripts that allow you to code-mode like experience in a safe way 2. agent-fs integration, imagine like gdrive, but re-built from first principles for agents (oss MIT too) 3. Pages & apps -> baascially artifacts. We've seen a company building a lovable inside an app (lol) 4. Extensions -> pi-mono like scripts that let you modify how the system works (e.g. we have jev reviewing the swarm replies to adhere to comms style)
What do you think about it? Do you think our vision makes sense? Roast me! :D
Cheers,
A few weeks ago, Boris posted how formal verification helped fix issues in their codebase (https://x.com/bcherny/status/2102543349102338309) and I definitely wanted to try it out in the swarm codebase.
The idea was simple:
1. Go over the codebase and fine most critical parts that could be modelled 2. Pick the first few and try to model them based on main code 3. Find counterexamples 4. Generate failing tests for them (single tests file diff pr) 5. Go over each of them and fix them + add the counterexamples in the repo for reference
I think this pattern is super nice tbh, and it helps a lot on reasoning around the code and implementation, specially when AI was involved.
The main issue: we had to add a daily wf to check for any updates to the spec, so that we keep it in sync (price you pay when the language can not be expressive enough, e.g. js).
Another win was that for the heartbeat system, which I wanted to refactor for a while, it managed to propose a simplification from 1.3M states to only ~4k (lol).
I believe it can be used for complex systems refactoring, while keeping the same functionalities.
Blog post about it (beware AI generated, but it contains some of the prompts we used during the session, just point your agent to the link and let it cook) -> https://www.agent-swarm.dev/blog/tla-plus-races-agent-swarm
I've been wanting to use tla+ for years, but always struggled w the syntax when working on something non-trivial. AI generating the specs and explaining the reasoning behind them helped a lot!
If you curious about the specs -> https://github.com/desplega-ai/agent-swarm/blob/main/specs/t...
Cheers,
I did a re-write of the post to remove the sloppy parts and making more clear some of the estimates and assumptions we made in the post.
Also linked this thread in there, so that anyone getting to that post can see all the interesting pieces mentioned in here.
Till the next on HN!!1
Then we know a few companies using it w +1k tasks x day (~20/30 people actively using it daily). Then we have some anonymous telemetry too, we are seeing more adoption, but as it's a toggle we only know about the companies that talk w us directly / contribute to the repo.
The type of startups, we generally see tech adopting initially due to the technical deployment required (e.g. k8s), but then tech managers, product, marketing and ops starting to use it more. Connecting integrations, and the Slack integration is super good, works as if it was a peer.
we did some iterations specially on the typing and error messages, as you mentioned, that help a lot in the reliability
But I believe it's not that common as it seems, only one actually trying to push this is executor.sh for example
it's really hard to measure some times. for example, once of the things we have seen is that the specific `workflow-triage` schedule we have have been running much faster and cheaper since we moved some parts of it to use scripts. From our analytics we saw times from ~5min before to ~1 or 2 min now. And in terms of costs we saw +50% reduction.
Why not 90%? because even if it uses the script, that's the "repeatable" part. we still have a reasoning around the output of it, and then deciding how to escalate, which removes the 40% reduction. Still 50% is nice for something we run daily.
in fact we are thinking a lot about this, not on a toll based, but rather on a role based.
giving access to specific tools to specific agents, how that works and evolves, and how that links w the humans using it
but it's not that simple. the whole point of this is that if you have an AI system that is in charge of this, what you need is a way for it to build those deterministic software based parts, while maintaining the AI routing and reasoning.
with time, our internal swarm has been building those "internal" software that is tedious and boring to build, where it makes sense.
e.g. we also have a "pages" thing (like claude arifacts) that it uses to join scripts, html and other components to build dashboards for us
Any recommendations on how I could make it better next time, so you do not feel like wasting your time (which I get and feel, dont get me wrong!)
Now our idea is to offer this to be harness agnostic (we support codex, claude code, pi, opencode, even devin). And you (as the one deploying AI systems) should be in control of the AI, the execution and the data. That's our goal
Also not exec happens in a sandboxed env, which means it might not have access to the fs by default (it has to agent-fs, which is a layer we built on top of s3 for agents)
Indeed we had the concept of workflows from the start (like n8n), but the point of the scripts is type safe executions that the agents can write, with access to APIs, MCPs, etc.
E.g. this PR from ~10 days ago https://github.com/desplega-ai/agent-swarm/pull/934 was going in this direction, offering a way to defined type safe connections to able to use more the scripts instead of adding MCPs or tools.
e.g. offering a way to do `ctx.api.gmail.searchEmail` coming from an OpenAPI spec.
The whole point of this blog is the "dumb" realisation of the following:
If you have an agent (say Claude Tag, or some other fancy Slack bot) that you keep asking the same things (e.g. check DataDog error and correlate PRs on a timeframe) and the agent does that process agentically, you are wasting tokens.
Now, if you give a way for the agents to write these scripts (code mode) the next time a task like that happens then essentially you just run that and you get easily 90% cost reduction in THAT process. Not globally, obviously.