Much better to look at cost per task - and good to see some benchmarks reporting this now.
Much better to look at cost per task - and good to see some benchmarks reporting this now.
Had to modify them a bit, mostly taking out the parts I didn’t want them doing instead of me. Sometimes they produced good results but mostly I found that they did just as well as the main agent while being way more verbose. A task to do a big hunt or to add a backend and frontend feature using two agents at once could result in 6-8 sizable Markdown documents.
Typically I find that just adding “act as a Senior Python engineer with experience in asyncio” or some such to be nearly as good.
If you delegate that work to a sub-agent, it does all the heavy lifting, then passes the results to the main agent. The sub-agent's context is used for all the work, not the main agent's.