Hacker News Reader

Comment by scosman | Hacker News Reader

Harness optimization is essential. Doesn’t matter how smart the model is if the tools you give it are ineffective, or the task is under defined. It needs invent the solution from first principles every time.

We did a harness optimization on Jev. Raw it was no where near as good as deekseek, but optimized it was significantly better and a tiny fraction of the cost. https://kiln.tech/blog/auto_optimizing_jev_with_autoresearch

View on HN

> Doesn’t matter how smart the model is

Is this true? I'd heard loosely that the higher intelligence models are less reliant on the harness; I'm still wading into the details of those claims so maybe someone more well versed can chime in, but my expectation was the current phase of harness engineering was likely only needed temporarily. I suppose turning Opus 5.5 loose in a variety of harnesses could be a decent if anecdotal test of that?

It depends on the goal. Given powerful tools, Opus 5.5 will eventually get the job done, but it might have to write/test its own custom software to get there. If you're running the same task thousands of times that's you want to move that work into the harness (building the right tools, right instructions, right guidance). Once you move it into the harness, you probably no longer need Opus, since the hard parts were pre-compiled into harness.

So for 1-off tasks: use smart model and powerful harness (bash + vm). For repeated tasks used optimized harness, and the cheapest model you can get away with.

Reply on news.ycombinator.com