adchurch

Really appreciate the thoughtful feedback!

1. Agree it's important, fwiw the proxy model doesn't blow this up though - only incurs a 1 time cost when switching models and we're aware of that when making routing decisions

2. The agents are model aware yes but they are not incentivized to optimize too heavily here (in particular they don't use OS models even when they would be better). I think that's where this router comes in and brings genuine improvement.

3. Two parts here: 1 is continuing to grow our golden dataset over time, 2 is using reward signals from production traffic (on a per-customer basis or, if allowed, across all users)

4. Yes we have these internally, great callout that we should publish! Will do + will link from the repo soon. (Fwiw I think these benchmarks are useful but don't fully capture vibes - you should try it out yourself for that!)

Appreciate the kind words! Lmk if you have any feedback on it from using!

I would argue they do not have a good incentive to build this and make it better. Why would Anthropic route Claude Code traffic to DeepSeek (at 20% of the cost)?

Very important consideration, addressed it in another thread (https://news.ycombinator.com/item?id=48689448). tl;dr we built this to be cache aware for exactly this reason

When we started building this we did it as an experiment and we thought the same thing might be true (cache misses would make the whole thing pointless). This turned out not to be true! I think there are 3 reasons intuitively:

1. Small models can carry out a good number of requests e2e 2. Small model for part of a request + cache miss < big model for entire request in many cases 3. Subagents

For our own usage we've saved 40% so far (that is of course including costs of uncached requests when switching models)

Fair enough, not meant to be marketing just a statement of fact. Would have turned me off too 18 months ago but times change...

It's a real concern! We take this stuff super seriously (https://trust.mycroft.io/weave) and tbh most of our customers opt for the hosted version because it's much simpler on their end + they're already trusting us with a bunch of sensitive data.

But of course since the source is available you can also run it locally or self host

Yep cache awareness is super important, mentioned this in another thread here: (https://news.ycombinator.com/item?id=48689448)

But intuitively I think it makes sense that a model can learn what model to route things to if it has all the relevant info, and experimentally it works pretty well in our experience

I guess delivering business value is always #1, I just meant it's the biggest problem they're trying to solve. Here's a recent example that was public: https://fortune.com/2026/05/26/uber-coo-ai-spending-tokens-c...

The choice on the first turn is super important for this reason! But if a user prompt sends the convo in a very different direction then often it does make sense to reroute at that point.

Nice, let me know any feedback you have from trying it out!

Very interesting - curious how you've used it yourself so far? I can imagine one use case would be having e.g. GPT 5.5 review Opus 4.8's work?

Yep 100%, mentioned this in another thread (https://news.ycombinator.com/item?id=48689448) but tl;dr we build the router to be cache aware

Indeed it is :) I skipped over talking about all the RL machinery, network design, reward function design, state representations, etc. because really the intuition is that we tell the model when it accomplishes its goal, and then it learns over time how to get better at making the right decisions in order to accomplish its goal.

Happy to talk about this in some more depth if there's anything specific you're curious about!

But we're not routing via `claude -p`, if you have sub usage available + it's the right choice to route to a Claude model, then the router is approximately a transparent passthrough. So it gets billed like normal `claude` usage rather than `claude -p`.

We integrate with OpenCode too! OpenCode provides the harness, then the router selects the right model for the task.

We haven't yet set up local model routing though, that's really interesting - have you had any success using local models for coding tasks? Tbh I haven't heard many success stories from using local models yet

100%, from what we've seen, for a lot of big companies that 1. don't have subsidized usage and 2. are pushing AI adoption hard, figuring out token costs is P0 or P1 for their eng leadership

Yep it uses the Claude sub if possible and falls back to API billing only if you don't have a Claude sub or it's out of usage! Same deal for Codex

I'd say that a typical main agent loop has 1-3 models (obviously very situationally dependent), but when you have subagents those can get routed independently since they have a fresh context window, so there are a lot more degrees of freedom there.

Yeah that's a really interesting point, tbh I think the more relevant variable here is the harness you're using rather than the specific model? i.e. GPT 5.5 in the Claude harness behaves a lot more like Claude than Codex if that makes sense.

Hard to quantify this ofc but that's what I've felt vibes wise from using this for the last month.

If you have a Claude/Codex subscription then we use that (and account for the subsidized price accordingly when making routing decisions) instead of API billing. So you get the best of both worlds: subsidized usage for frontier models + save by using open/smaller models when it's genuinely better.

In practice, lots of ppl are using this to make their Claude sub limits go further!

Oh cool, feel free to reach out to me at andrew@workweave.ai if you ever want to share notes! We've learned a lot in the process of building this so far :)

If you statelessly route each new request: yes it does end up being more expensive!

So our routing is cache-aware. It will have a much higher threshold to switch from one model to another if there's already some cache for the first model. Experimentally this solves the problem (like I said we've saved 40% ourselves vs. what we would have otherwise paid).

Fun fact: Cursor's "auto" mode is just Composer (or at least it was last time I checked). So it's different in the sense that it actually does route to more than 1 model

It saves money because some agent sessions can be entirely handled by a smaller model (also relevant: subagents use fresh context windows so a subagent with a simple task can be routed to a smaller model even if the main agent needs a frontier model).

Totally right about small LLMs btw, that's why we trained this on real agent sessions where we forced it to use different models. If the routing model sees small models can't handle a certain type of task then they won't be assigned. (Also as a fallback we have some guardrails that will have a bigger model come in to "rescue" a smaller model if it gets stuck)

You're right and that's why we built the router to be cache aware! Once it starts using one model, the threshold to switch to another model will be higher because the additional cost of the cache miss needs to be worth the cost savings or quality increase.

This is the key thing that other routers we've seen miss: they're stateless so for a coding agent use case you end up spending more money due to all the cache misses.

Author here, great to see all the conversation & thoughts you all have shared so far!

One thing I've seen a lot of people saying: hiring juniors isn't worth it because they'll just leave for more money in a couple years.

I got hired with 0 experience for my first job and stayed for 3.5 years (I left when I decided to start my own company). I was otherwise never tempted to leave because I got all the support and growth opportunities I could have hoped for, and I always felt fairly compensated.

So based on my own experience, it's possible for a company to treat people well enough that they won't just leave. I do think software companies tend to be bad at this specific thing though - I'd imagine many have had an experience unlike my own.

If you keep asking questions eventually you hit something they don't actually know. How long it takes to get there + what they say when they get there is what makes this interesting imo!

Great idea re: giving hard problems. Same motivation behind why we ask people about past projects and keep diving deeper and deeper. The point is to figure out if they're curious & capable of engaging on a deeper level, vs. just following a tutorial they found somewhere.

Check out the bit about the hiring process, particularly about filtering for the right mindset.