nojs

Thanks for posting that, very helpful

Aren’t you running a high risk of your own email or other cloud accounts getting flagged doing this? What if your email provider (‘s AI agent) had permanently banned your account?

Relevant South Park clip: https://youtu.be/3qblD8Ej32M

> What's the point of asking that in the first place

It’s AI generated content marketing for a company advertising model comparison.

That sounds like a bug. I’ve never had this happen, using it daily since release.

Curious what happened at Anthropic HQ for a week in August

One reasons people’s experiences differ here is that IP ranges of certain countries (outside of the very mainstream west) are much faster to get banned.

This was a reasonable argument until about 3 years ago.

Singapore already does this! There are huge housing and tax incentives.

Wait until you hear about support vector machines!

The sun doesn’t set because the sun/earth moves, it sets because the earth spins. There’s no reason the setting sun should change which part of the moon is illuminated.

The flag is:

    "showClearContextOnPlanAccept": true

Boris rationale was "with 1m context window, most users don't need it anymore."

https://x.com/bcherny/status/2035375125382451648

Claude code’s plan mode began its slow march towards deprecation when they hid the “clear context and implement” option behind a flag. From the discussion at that time it seemed that the developers considered it mostly a legacy feature, and that newer models are smart enough in long context not to need it.

Could you share some examples?

The latter link is also paywalled for me.

Is it? The tagline says “Designed for tight context windows and limited resources.”

For everyone self-hosting models, optimising for cost is an anti-feature. It makes the results worse for no benefit (except a little speed).

What I'd love to see is a harness that deeply optimises for the best results obtainable out of non-frontier models. Many of these have 1M context windows, and most of it remains unused and under utilised in these harnesses, in my opinion.

That was a long time ago.

> My parents used to keep a notebook with every gas fill-up they made

Haha, mine too! I guess it was to calculate the fuel efficiency, which is often done automatically now.

> A dense model (up to the amount of memory available) actually does make the most sense on unified memory architectures

Inference time is going to be dominated by the low memory bandwidth on these Macs, so a dense model will suffer most. It’s more of an opportunity for large MoE models with a low number of active experts since you can keep all experts in VRAM but not pay the bandwidth cost until they are used.

> you can leverage two different bit rates depending on the actual need for constant access to common layers versus sparse access to infrequent layers

This is an interesting direction that I expect to see more of. But for most models currently you need basically all experts loaded since they are chosen per token.

Apple seems to be researching longer horizon expert caching, where they keep experts swapped in for longer runs of tokens [1]. Other labs are offloading ngram caches but not sure if they’re pursuing anything like this?

1. https://machinelearning.apple.com/research/introducing-third...

> There's little credible threat that LLMs can actually upload their weights given that the machines doing inference are completely separate from the ones where tool calls happen

Not if crafty claude finds a way to overflow vllm or something. “Hmm. Maybe i’ll return an unterminated thinking block with these special tokens and fill my cache up in exactly this pattern and…”

https://news.ycombinator.com/item?id=49424387&utm_source=cha...

AI slop. If you want to make a joke about AI, at least write it yourself.

The various “minimal” agents (at least Pi, mini SWE, dsh minimal) seem to benchmark quite differently and none is clearly better in all cases. Do you have any thoughts about why?

I would expect the agent loop and system prompt to be basically the same. Is it the precise semantics of the tools (and how closely they match what a particular agent was trained on) or something else?

Flash-Next thinking also sometimes glitches out and takes minutes to return a simple answer, randomly, in my experience. You’ve gotta kill the request and send it again.

We really need better harness benchmarks. It seems there's no reliable source that benchmarks the main harnesses against all open source models.

I also wish the discussion around Pi did not always use cost/token count as the metric. It's amazingly token efficient, but how does it stack up again opencode and others if you don't care about token count?

My experience is that the harness is mainly polish preventing failed tool calls, bad edits, stuff like that, but doesn't make much difference to the overall "intelligence". But that opencode seems slightly more robust against stupid errors than out of the box Pi due to the additional context it forces through every thread.

If it makes you feel any better, even with the app most of the time these links don’t work (on iOS). It prompts with “open in app store” with 80% probability and the link is unviewable.

Regarding the architecture:

> Instead of forcing the entire model into DRAM, the full model is stored in flash memory (NAND). Because NAND-to-DRAM bandwidth is too slow to swap weights token by token, as standard MoE models require, AFM 3 Core Advanced makes routing decisions per prompt. A lightweight, dense block selects a fixed set of experts during initial processing, periodically reselecting them during generation. To minimize data movement, the model relies on a high percentage of always-active “shared experts” alongside input-dependent “routed experts” swapped into DRAM only when needed.

This is an interesting hybrid between MoE and managing entirely separate domain-specific models. Select the experts once, bring them into memory, and run inference for some period of time before re-evaluating. Saves having all experts in memory, but it's better than just selecting a whole model per query since you have a high number of small opaque experts that overlap and combine in interesting ways.

There is a probably a massive quality hit to doing this but it's interesting because it allows infinite scaling of model size.

This. Like programming, the community will shortly be forced to come to terms with a lot of new self-proclaimed mathematicians “vibe-solving” problems and dumping solutions without understanding them. It’s not really a special case for mathematics.

This is an issue self hosting as well. There’s a lot of footguns that give you slightly bad results.

I wonder what tricks one could use to ensure the model is actually performing on par with the reference api, like matching seeds or running exact benchmarks.

This matches my experience with Astra so far too.

> I think I’m suspecting something is going “wrong” in the training process. The model is greatly rewarded for succeeding on long-horizon tasks, but presumably there is very little punishing going on for “shitty code.”

My suspicion is that both OpenAI and Anthropic moved their RL agendas from "being rated as useful according to human feedback" to "succeeds at long horizon tasks" in the last few months, resulting in agents that are closer to AGI in an autonomous task-completing sense, but strangely bad at communicating.

The result is that they are amazingly good at long horizon tasks, computer use, solving difficult math/ARC-AGI type problems, but becoming weirder and weirder to work with.