How do you measure whether your coding agent is becoming more effective?

1 pointmanytimesfail1 comment

I do my best to clean my repository, remove bad code examples, create lint rules, provide more timely information for my agents, etc... but I haven't figured out how to measure the improvements. Maybe it's moot and measuring the time between the model I'm using and the next is the only stat that matters... how do ya'll think about that problem?

View on HN

Question is, what is more effective? Using less tokens, writing cleaner code or, and I think that one matters most, number of inline comments on a PR the agent created that got actually fixed, ie defects. The time and token that go into the review/push cycle before merging. I personally would focus on the latter.

Reply on news.ycombinator.com