Hacker News Reader

Comment by fakwandi_priv | Hacker News Reader

Slightly off-topic to this model card but has anyone noticed changes to GPT6 Astra about a week after 6.1 came out?

I feel like 6 Astra just turned a lot lazier during planning and strangely enough claude has become a lot more assertive with Opus 5.5 while it used to be very lazy.

It’s really obvious during planning, it used to be that claude would just give a best effort guess and handwave things but now it will actually write small tests, launch playwright to test a hypothetical and even look things up in github during a question. This was something codex’s models were really good at.

I’ve read so many stories about people claiming models getting nerfed after a new release and now it seems I can finally join them.

View on HN

IMO Astra isn’t really lazy, just extremely literal so when you use it side by side with gpt-6.1-sol, it might appear more lazy but it’s often just the big difference in prompting between the 2. Astra wants a clear goal definition of what you want it to do, while sol infers a bit more. And the Claude’s infer a lot lot more.

From memory, prompts like “this file is a bit messy” would trigger Sol and Claude to start editing the file immediately, while Astra wouldn’t. But if you give Astra a clear plan and a definition of what you want it to do, then it will keep working until that ask is fulfilled.

Especially compared with Claude that’s a huge difference, Claude is so proactive, I have to ESC it multiple times during the day and tell it to wait because I only asked a simple question.

In my experience the failure mode is more broad than just lazy, the issue is:

* It doesn't do things you ask it to

* It adds things you didn't ask it to

It still does both and therefore I still consider it failing. If you don't want to call it "lazy"; fine, but it seems as though the underlying issue is still there.

Claude does this too, FWIW.

This frustrates me as well. Especially when I set a goal and come back, ask what went wrong, and the model hits me with something like “sorry i didn’t do that, gave up when x happened” and then I ask if x actually happened and the response is like “oops no I just assumed.” I had similar issues with older versions of sol at the ultra effort level. Lowering it seemed to help then, but I wish I had better insight for getting Astra to work better without e.g. having another LLM check its progress every hour.

For that claude problem in specific about 95% of the time it will follow the rule that a plan file needs to be made before any changes to the project are allowed. Haven't had to ESC it in a while now during a question, things definitely improved in 5.5.

I imagine this is basically not at all due to actual model changes.

If you use Codex, Claude Code, etc, those are being constantly updated. That sort of behavior would be highly harness dependent.

I just know that over the weekend, I was more wtf is going on with either Astra or Opus 5.5 than usually.

My guess is models are constantly tweaked behind the scenes to adjust to available resources, steer people to other models, accommodate usage on other models, try out optimizations, etc.

Performance seems to vary like the weather.

I think you also likely have three letter agencies involved heavily now. The NSA is, without a doubt, policing what can be offered to the public and ensuring what the NSA has access to is a step or two beyond what is public.

I'm one of those people who thinks the models aren't improving.

5.5 Sol would anticipate what I wanted and get it right more than often than not. 5.6 Sol kept anticipating, but would do things that made no sense in context. 6.1 Sol seems to like that but even more so. It spends a lot of time thinking, but the quality of the decisions isn't outstanding.

Astra was always lazy as a default. I wrote about this >1 month ago https://srcreigh.ca/posts/gpt-6-astra-wont-try-hard-unless-y...

Reply on news.ycombinator.com