Do you have task completion numbers alongside the reduction in tool calls? I’d be interested in whether narrowing the tool set ever removes something the agent needs later in a task.
I’m building Gear, an open-source framework for evaluating and improving agent harnesses. One result that made aggregate metrics less reassuring: a harness change took our pass rate from 36% to 40%, but underneath that were 14 tasks improving and 10 regressing. I’d want to see that breakdown alongside the tool-call savings.
The experiment is documented here:
https://github.com/rsi-gear/gear/blob/main/docs/guide/en/exa...