Hacker News Reader

Comment by sigmoid10 | Hacker News Reader

Which model/agent/harness tool did you use? I've found what you describe was my exact experience some ~6-8 months ago, but since about a month or so the game has completely changed. Using 5.6 Sol with highest reasoning setting in Codex or Fable in Code, the models come up with a list of possible improvements from static analysis (ranked by complexity/benefit), write and run their own custom profilers and deliver significant performance improvements with barely any input needed from my side. So this is no longer a model issue, it's a user toolchain issue.

View on HN

Very hard to say anything definitive on this because it's a moving target, but last time I tried models still had a distinct sense of "consistently good, sometimes great at micro, bad at macro". Similar to how, even for relatively pedestrian CRUD, they'll do code that's objectively fine at the function/file/class level but can still make a mess if you don't supervise them at least at a high-level.

It is moving so fast that your experience with older models is irrelevant today. Sorry.

Claude Code, Fable 5, xhigh reasoning, allow it to run the full CI, end to end and benchmark, it will not make silly mistakes (or only occasionally). Also, be able to state what you desire. Have any docs or materials in the same directory so the model can reference it. For even better results: turn on speech recognition and braindump what you know about the system, its goals, its context, history anything relevant, any gotchas you'd explain to a new employee or intern. Talk for 5-10 minutes. This is optional, "make it faster" can already get a large part of the job done.

And if it doesn't work well, describe what you dislike in its solution and tell it. Even just one extra iteration can make things work.

(I guess GPT-5.6 can be similarly good, I use Claude)

I feel like some people are emotionally invested in it not working and subconsciously sabotage their own effective use of the tool.

I'm not saying I don't get any use out of them, fwiw. In a lot of ways they write perfectly fine code, I'm just saying without some guidance at the macro-level they lose the forest for the trees, that's all.

Yes, definitely. It's too happy to just go and diligently work based on too little information, instead of asking questions first. And it makes many people think that they don't have to specify what they actually want, and the AI will mindread them and do what they want without being given access to docs and tools and context info that's crucial to know and they would intuitively know to hand over to a human collaborator but with AI they think they don't have to.

Which model and harness did you try?

5.6 Sol on Codex, Opus 5 on Claude Code. (See sibling message, I'm not saying they're useless)

Then it's for sure a guidance or missing context issue. When given full access to everything they need, these models deliver results on par or above the best coders I've ever met and they are lightyears ahead of the lower 90% of developers. I feel more and more that when users report they can't solve their problems this way, it's like when a gorilla is mad at Einstein after talking to him and he didn't make bananas grow instantly.

> guidance or missing context issue

I mean yeah, that's mostly what I'm saying?

The only parts where I would disagree are that capabilities are spiky, and also the fact that they're a bit prone to losing the forest for the trees without guidance (the macro/micro distinction). But yeah, by default I don't write code by hand.

Reply on news.ycombinator.com