Hacker News Reader

So, IMO, they have much better capability internally because they have 'unlimited' usage. I also suspect a lot of 'Fable' is just a shit ton of nested subagents doing necessary backend work with a very solid fine tuned LLM conversing with the user.

I worked recently for a German megacorp, for whatever reason they gave us unlimited access to a bunch of models, up to sonnet 4.5/Gpt o1 etc, so not the best or latest models. And no agentic tools.

So I built a custom Claude Code clone because they exposed the api from their custom 'LLM Chat' clone in the client and accepted my user credentials for access.

Essentially a frankenstein AI harness using various models for their strengths, including using the OpenAI best model for memory context consolidation and compaction + Sonnet as orchestrator, etc. I was able to spawn 20+ agents to do my work and basically automate my entire job at that corp. I think my internal bill was something like 5k euros per month and I was being careful to not abuse it...

It was better than the best you could get from claude at the time (about a year ago now I think), in terms of task performance completion.

So imagine what someone with essentially infinite spend can do with endless fable agents. AKA US government and Anthropic internal.

> I was able to spawn 20+ agents to do my work and basically automate my entire job at that corp

Not a great plan

Not if you want to stay at the job. I quit to work full time on harness development.

whats your harness called?

I guess doing pantomimes for 'job security' has become so accepted and mainstream that it doesn't rub people the wrong way.

I'd rather just do the best job I possibly can, with the tools available.

I need money, but I also can’t live with myself if I know I’m doing something the wrong way.

The thing that bothers me is the false advertising. The models are announced with specific capabilities and benchmark scores. Then they silently nerf the models after release. Without a public acknowledgement that a production model has been nerfed and what the new benchmarks are, people are assuming the model has the original capabilities. People are choosing to pay for models based on their reported capabilities. Customers trust these models to deliver the reported capability. It feels like false advertising to me when they say one thing and deliver something less silently hoping no one finds out.

If I order a 16oz beer and get a 12oz pour, I'm going to call it out.

I wonder if the questions about the quality in claude code have something to do with it. Like as long as you say it's great, they can take little steps dialing the effort back.

> If I order a 16oz beer and get a 12oz pour, I'm going to call it out.

And then Sam Altman is gonna sue your ass for owning a measuring cup.

It's kind of crazy they can do this without informing customers. Essentially we like many other knowledge working companies are slowly outsourcing intelligence to a 3rd party that can, at will it seems, alter that intelligence without informing us. Just means we have to invest more in harnesses, guardrails and keep active in open model space.

Google isn't informing customers everytime it updates its algorithm

1. Two things can suck independently at the same time. One thing sucking is not a justification for another thing sucking.

2. Google doesn't pretend that its algorithm has certain capabilities while serving the user a nerfed version

I think (2) is the important one. Anthropic provides benchmarks with their model releases. If they are making changes that make models perform worse on advertised benchmarks, how can this be legal?

It's a good point. I mean I have no idea if they are actually breaking the law, would assume their tc's would dissolve them from any legal jeopardy but it's still not right.

Is this where independent benchmark companies like artificial analysis could help ?

In fact, people accepting that the new thing can suck is why newer things suck more. That's how the frog boils.

Google gives a lot of information to its customers, the advertisers, when it changes things that impact them a lot. Not enough for all of them to be happy, but it does private and public communications on big changes.

caveat emptor

But like, you wouldn't say that about other services where we trust the provider out of necessity, like engineering or pharma.

Seems to me that imposing reputational consequences on supply shenanigans is one of the key ways that an emptor may cavere.

It would be nice if Anthropic would be more transparent about these things.

I'll gladly sign up to be a "Experimental Settings" beta tester (with knobs of how extreme I want to go) for a slight discount (or just bill me the same, just gift me a reset every week? idk whatever).

Doing A/B testing in prod on a frontier model that could really mess with the users in really bad ways is a bad idea.

Why would they ever do this instead of just rolling things out like they are now? What is the incentive to give you anything when they can move faster and just get the same result without it?

Because sometimes when they do these things some users are affected for weeks and leave for competitors, I guess they don't care as long as they don't have to be beholden to stock investors, but eventually that too will matter.

Don't you think that this is a calculated trade-off? You can roll out the A/B testing to 2%, or 10%, or 50%. There are statistics on all this. Individual user feedback is irrelevant.

> Individual user feedback is irrelevant.

Like Windows 8?

Windows 8 is pre-big data and massive scale data collection of every aspect of the human.

They had do rely on this kind of feedback. Now this is all automated, engagement goes down -- something should be fixed

> they secretly have a huge amount in the tank they aren't releasing

I think it's more how much immediate profit they are willing to forgo today per account age, subscription tier, and usage type. Burning more compute means they make less or nothing (or have to subsidize) vs dialing up increasing amounts of the ever-more aggressive 'optimizations' (aka nerfs) required to get them closer to viable profitability after their IPOs.

They have dozens of 'knobs' which are completely opaque to us yet each indirectly trades worse results for lower cost. On the other side, they MUST show a clear path to HUGE top-line revenue growth and bottom line profit growth from where they are today by min/maxing new account retention AND renewals-after-price-increases and continual nerfs.

We're not using a somewhat variable, load balanced cloud service, we're in a carefully engineered Pavlovian Monte Carlo casino. There are at least 1,000 different Claudes, Sols and Astras at any given moment. Every time we start a session we're spinning a behavioral roulette wheel. In effect, we're the crabs in a pot of water that's being pushed ever-closer to boiling while they're testing to ensure not too many of us jump out per degree increase (they're fine if the less profitable ones jump out). This will inevitably worsen since they are still a long way from justifying their valuations.

I feel like I am taking crazy pills when reading the other responses, but come on, this seems like normal human psychology when being exposed to a new thing over and over. It feels new, it solves everything, but then over time you get bored with it. Attempting to proxy it with "account freshness" just masks the problem. It's the reason why people tried to quantify it with stuff like https://github.com/ninjahawk/livenerf because the pattern you are referring to is a known bias.

https://www.reddit.com/r/ClaudeCode/comments/1xr9x5y/the_ner... ---> Interesting reddit thread going over what appears to be evidence of this kind of behavior from Anthropic.

You're telling me HN users have biases just like everyone else and engage in conspiratorial thinking or reverse engineering "rational" justifications for whatever they're feeling in the moment? no way ... /s

I noticed an enormous drop in quality around this time.

Opus 5.5 was significantly crushing every task I gave it for quite some time.

Suddenly, it starts making the type of mistakes that even Sol 5.6 would not make. It became noticeably worse than Astra.

I've grown so completely tired of the labs changing the quality of the model out from under our feet that I CANNOT wait until there's viable alternatives at the frontier level.

I will gladly forgo the best available power for consistency - and I see that coming soon (hopefully with Qwen 4 or shortly after).

DeepSeek v4.1 Flash is not even remotely in the same league as an un-nerfed Astra or Opus 5.5 - despite what microbenchmarks may claim.

It's a great model, especially for the cost - but there's a huge difference.

Same. I'm doing something security related right now and I swear it was smarter than this before. Right now it keeps pushing me to just allow all connections, or leave a password in plain text, or other insane suggestions. Maybe placebo, maybe bad timing though, but it feels ridiculous.

Edit: Just to add some more ridiculousness:

> The last squash stalled because it opened an editor to merge descriptions. I've stopped it. Checking what landed:

I was using jj with Claude for a while now, suddenly it forgot how to squash? It also started doing that stupid thing where it edits files through python scripts so I don't see the diff during manual mode. This is like 3.5 levels of stupid.

If this is in Claude Code, check whether your session got switched to Opus 4.8. After a cyber safeguards refusal it continues on claude-opus-4-8 with a one-line notice. I saw this on ordinary auth-token fixes, in 5 of 60 runs last week.

That is a setting. The first time you get refused it offers to stop or switch to 4.8 and if you choose 4.8 it automatically downgrades silently whenever it happens again. I just shut it down and switch to Codex when this happens.

I would also agree that 5.5 feels dumber the past week. More weird logical problems. it's not as dumb as 5.1 or 4.8 but it feels like it regressed.

I'd started looking at https://marginlab.ai/trackers/claude-code-historical-perform... this week because doing the same tasks a fortnight ago versus this week and even today I was having to argue with or question everything from Opus 5.5 on medium.

The link hasn't been updated since September but anecdotally I had Claude today when asked to write something create a 'to-do' and instructions for me to write a code block.

Another example is suggesting tailscale serve to proxy from one address:port to a different port (so still address:port) when setting up a reverse proxy.

> Either as a card to play in the future, or because they can't keep up with demand.

I suspect it's rather a simple matter of cost.

Fable 5.2 just dropped!

https://news.ycombinator.com/item?id=50429947

Whether true or not, this kind of sentiment will eventually destroy their business model with users preferring local models instead. When they are good enough.

They probably keep it at "full power" when released because that's when most people will be running benchmarks, then cripple it to save money.

In a just world this would be considered consumer fraud, or defrauding investors. Things that if you read the law you would imagine that it came with tens or hundreds of millions of dollars in fines and/or jail time for everyone up and down the chain.

If you're selling model capabilities for money, and you change the mobile capabilities, not changing the model designator (name and/or version number) is fraud, full stop.

A VW engineer got jail time for Dieselgate, right?

Hopefully that was a manager and not an engineer otherwise I'm fearful of doing any engineering in ${conviction location} (Germany?)

Hopefully it was every involved from top to bottom. Crime is crime.

And yes, if you're doing engineering that violates the law, you should be fearful of doing. That's kind of the point of punishing criminals.

> A Volkswagen engineer was sentenced today by U.S. District Judge Sean F. Cox of the Eastern District of Michigan to 40 months in federal prison, and two years of supervised release, for his role in a nearly 10-year conspiracy to defraud U.S. regulators and Volkswagen customers

https://www.justice.gov/archives/opa/pr/volkswagen-engineer-...

Mindblowingly, the CEO and 4 execs.

Unlike here where the "phantom bank accounts" fraud was successfully pinned on tellers and branch managers who were all "independently" just trying to meet management goals...

DeepSeek flash doesnt have this problem. Im routinely astounded how much i was being gypoed

It's more likely it follows the compute demands of the development cycle.

> Every time a new model drops it feels wildly powerful, then regresses to some sort of generic SOTA level we've been used to over the last 6 months.

Is this 2026's "my macbook feels 'snappy' when I update it", though? (That is to say, not really much in evidence)

House of El covered this in her latest video incidentally.

> I don't have any evidence/study to substantiate this but I have the feeling there's a _massive_ amount of fiddling with the quant/functionality of the models at different stages of the release cycle and account freshness.

Oh good Lord, they stir the pots constantly. Anthropic is hella naughty about this. Someone else on HN said it well, Anthropic has too much of a "researcher" mindset. You cant experiment on human populations tool use the same way you run AI/ML experiments.

I want a stable tool that works consistently, not a hammer that changes mid-swing (credit Mario Zechner for this analogy).

> You cant experiment on human populations tool use the same way you run AI/ML experiments.

Google and Facebook have proven otherwise. Unless you are very, very lucky you are always part of multiple running experiments on their products, and nobody really cares.

If it's mostly indiscernible from baseline that's perfectly fine and allows for incremental improvement. Think infinitesimal changes in light of multiple dimensions. When you have separate development and stable versions there's always the risk of packing too many large changes and no one actually wanting to upgrade (given a choice) because the product is already good enough. Pro and con for both approaches but IMO, you need mostly small changes and occasional big steps when small ones can't get you there.

There's direct evidence - OpenAI employees on twitter (tibo) flatly say they are constantly adjusting models after release, and even specifically state "improvements" to efficiency etc for models post-release. They presumably release to maximize benchmarks at the start, then make them "more efficient", and if you want to give them the benefit of the doubt, then presumably more efficient with negligible loss in benchmark scores.

For a while, Codex was accidentally truncating its thinking at multiples of 516 tokens: https://github.com/openai/codex/issues/29353

Because the codex harness and the intelligence tests are separate tools, you can end up in situations like this where A/B tests inadvertently break the tool customers actually use, while the tests say everything is awesome.

I bet some engineer got a bonus for “halving token usage” through “efficiency improvements” — announced around the same time — and not realising that they’ve accidentally lobotomised their model in the process.

“Oops.”

Also, it becomes really difficult to unwind errors like this because the “savings” get burned into profit and loss reports, S-1 filings, and so on.

Can you imagine that boardroom conversation!?

I had assumed this was just me. I keep seeing the models do things nearly magical upfront, and then regress is subsequent weeks.

Last night I was working on datalog, but Fable was acting stupid and its responses were LLM babel. After every response, my reaction was "What?". I stopped using it. This morning I wake up to this in my Claude Code emacs session: "Why are you mixing up the datalog stuff in here? you're working on emacs." Claude Code replied, "I did the work the datalog session sent here instead of passing it to you."

Volkswagens new diesel powered LLM has great environmental statistics.

> Every time a new model drops it feels wildly powerful, then regresses to some sort of generic SOTA level we've been used to over the last 6 months.

This isn't new to the current LLM bubble. Grammarly has been like this for 10 years.

I agree, it seems like they try to find optimization twoards a baseline short and long term, and maybe one for transition time for a new model.

I'd venture to say they are about to drop Fable 5.5 or perhaps the next Opus version. Over the weekend, their current models were objectively much slower than usual and subjectively not very bright.

I don't know who they think they're fooling.

Edit: Fable 5.2, whatever. There you go.