Hacker News Reader

I'll write something up, what are you curious about exactly?

> Do you have documentation on how you built the whole system

Or any actual "proof" (i.e. source code) that your method is useful? I have seen a hundred articles like this one and, surprise!, no one ever posts source code that would confirm the results.

I have been trying to figure out how to publish evals or benchmarks for this.

But where can I get high quality data of codebases, prompts, and expected results? How do I benchmark one codebase output vs another?

Would love any tips from the HN community

That's the problem with people who use AI. You think too much and fail to deliver. I'm not asking for benchmarks or complicated stuff, I want source code, actual proof that I can diff myself. Also that's why the SWE is doomed because of AI, but that's another story.

the implementations of getFileContext() and shouldStartNewGroup().

Reply on news.ycombinator.com