> given a sufficiently articulated prompt
Are these two statements not at odds with each other? You can always say a prompt is not sufficiently articulated if you need to add additional handholding and vice versa, thereby invalidating any meaning of measurement.
then a PR that tends to only get minor feedback is produced
As in, many if not most reviewers have always been nothing more than approvers, i.e. people that click the approve button with minimal to zero review, maybe a "pro forma" comment so nobody has them show up in some statistic as "only clicks the approve button".
With the amount of AI generated PRs this will have tended to become more of a problem, not less.
TLDR, the coding output and automated reviewers have superseded humans at this point. No human is reading the PRs closely, anymore.
But sol 6.1 is at least in the conversation as being as good and is much cheaper. I farm out bunches of my tasks to Luna and Haiku these days and have a working prototype of a glm 5.3 workflow at work that shows it being competitive.
The price pressure is very real and if you aren’t on a subscription then there is virtually no reason to pay the Anthropic premium
Not due to capability per se, but writing style and polish is much worse.
It will leave really bad comments in the code, commit messages etc without strong explicit steering. Though I've basically moved away from having the model write any documentation at all.
It also tends to talk very circuitously and referencing things implicitly instead of just being direct and clear.
Certainly it's more capable in debugging, but I much prefer Astra over Sol now.
Of course, Astra has its own problems of code golfing everything. But at least it's very concise and direct
There's definitely unique things that so far only Opus is capable of doing.