Hacker News Reader

Comment by nico | Hacker News Reader

> with 184 tokens/s on an Apple M5 Max

That seems pretty decent. How many tokens/s do you need for a coding agent to feel like Claude or codex?

Claude code is 60-80 tps, codex varies by model but mostly 40-50 tps.

200-250 is the rate I've seen my context window fill up during longer tasks with Claude. The 40-80 figure is what Google's AI mode says, but I don't know where it got that from.

I've been messing around low computing power hardware to run models locally and to me 100 is already plenty fast. You most likely won't notice a huge difference between 184 and 250, unless you're running some serious multi-age agent circus.

The context window fills up also/mostly from prefil which is supposed to be 20+ times faster or so. I think 40-80 tps for decode/output is believable for claude models

>How many tokens/s do you need for a coding agent to feel like Claude or codex?

it's kind of a loaded question, this model gets a lot of tok/s but reasoning is forced on; the time-to-answer is really a combination of things like "how much does this thing want to over-reason" and "how much does this thing like tool use?" .

You are right, they are not exactly the same thing, just wanted to have a reference point

When running locally usually I have to trade off speed for a better model. I can run decently big models at about 2tk/s, or a pretty dumb one at 40-100tk/s

fwiw this thing puts out results very fast even on max reasoning.

the results i've gotten so far have been a little braindead though. still playing around. bad luck with traversing filesystems remotely so far.

it seems to want to make concluding decisions very fast regardless of reasoning levels.

Another data point: I got 94.74 tokens/s on an M4 Pro Mac mini 64GB with the simonw prompt "Generate an SVG of a pelican riding a bicycle". TTFT was 0.37s. The model thought for 40.75s before answering.

(The resulting pelican was underwhelming.)

AFAIU prefill speed makes a bigger difference in agentic coding workflows as the harness likely spends more time reading than creating new code. Qwen in particular spends a lot of time reasoning so it does make a larger impact there

I’d say only 65

Reply on news.ycombinator.com