Comment by sadilek | Hacker News Reader

Comment by sadilek | Hacker News Reader

Parent Full thread

Haha, well, I don't because I have no reason to assume this is in the code. In fact, I have reason to assume it is not in the code because the models aren't trained to produce this.

Could there be a less egregious version of that in the code? Sure, but then that's just a normal bug where implementation doesn't match intention, and that happens to human developers as well. I find that Opus 5.5 is so good that it writes pretty good code that follows YAGNI, KISS, DRY, and is fine with writing a lot of tests, so my sense is that the rate of normal bugs is actually lower than with human developers.

Now, if you're assuming malintent on the side of the model creators, well, that's another story. FWIW, in a commercial project, where I also work at the "engineering director level" since Opus 5.5, all non-trivial changes go through reviews by Codex and Grok (locally) and Greptile (on the PR), so there would have to be some serious level of collusion by the labs to produce your example ;)

Well, I just used a contrived example to illustrate my point.

You would be well served to review the code. I don't usually need to do much of that since I work with models edit by edit, with every single line they write approved by me.

You would be surprised at all the errors even Claude Opus 5.5 commits. Not syntax or even basic logic errors, but more insidious errors that betray a lack of being able to see the bigger picture, or serious architectural errors.

If you want example, I have them in abundance. These models still need serious supervision. Unless you are spot checking the code deeply, you won't notice.

"I work with models edit by edit" – That could be the cause of bias: Maybe because you're supervising small step iterations of the model, you notice issues that the model would have noticed itself if you had let it tackle a larger scope by itself. Worth a try in my experience. And the extra review agents make a big difference. Even if it's Opus reviewing Opus. Just by having a fresh context window, the model finds issues it created itself in another session. And with a different model, you get yet another fresh perspective. I find Codex is particularly thorough in reviews.

No, the model will not notice.

How do I know? It several times present deep architectural decisions to me, where each options may seem plausibly good. I rely on my deep architectural knowledge and experience, and knowledge of my vision for the product to steer it to the right option.

Reply on news.ycombinator.com