Hacker News Reader

Comment by criley2 | Hacker News Reader

What would you prefer, 30t/s 50k output tokens needed for solution, or 300t/s, 500k output tokens needed for solution?

This is why "tokens per second" alone is a deeply flawed comparator.

This is also why discussions of Chinese models focus on tokens per second- they struggle to compete on intelligence so they dramatically increase effort levels to compensate, meaning they require massive total output tokens.

For example, flash 1017 isn't benchmarked yet, but 0731 required 15.4M tokens to complete the artificial analysis benchmark, and Sol 6.1 high required just 2.8M.

Sol 6.1 uses 2.8M at 64 tok/s = 12h 9m

Flash 0731 reported 149 token/s at 15.4M needed = 28h 42m

And in my example in the first line, scenario A (30 t/s) completes 6 minutes faster than scenario B, despite 10X slower inference.

View on HN

> What would you prefer, 30t/s 50k output tokens needed for solution, or 300t/s, 500k output tokens needed for solution?

The best way to answer this question: [Pareto frontier or cost/per task curve](https://api-docs.deepseek.com/img/v4.1_261017_perf_cost_en.p...)

img from https://api-docs.deepseek.com/news/news261017/

I certainly don't think the self-reported single benchmark scores are the best way to answer the question. I waited for the Artifical Analysis run (which itself is still mediocre, but at least it's an order of magnitude better), and predictably the results tell a different story.

Sol 6.1 High: 50 intelligence, $0.32 per task, 4.4 min per task, 13k output tokens per task

Sol 6.1 Medium: 48 intelligence, $0.21 per task, 2.9 min per task, 8k output tokens per task

Deepseek 4.1 1017: 45 intelligence, $0.26 per task, 4.8 min per task, 89k output tokens per task

If you look at the actual pareto frontier across the full benchmark, Sol 6.1 remains firmly on the pareto line, while deepseek remains firmly under it, just as it was before https://i.imgur.com/N72p5NF.png

This result is exactly as I expected, and seems to run counter to the effusive praise of this thread. Sol 6.1 Medium is producing higher quality output at a lower price with nearly 10X less output tokens! And this is not surprising.

And finally, Mimo V2.6 Pro continues to look like a stunning release, clearly smarter, cheaper and faster than deepseek v4.1 1017

"Sol 6.1 Medium is producing higher quality output ..." -- When it is actually up and working. Even last night I still received constant capacity messages halting codex over a dozen times and I gave up and shut the workflows down.

I would prefer it work and solve quicker. Opus 5.5 has a reasonable pace to me. Sol does not. I understand efficiency I just don't like using it right now because it has literally hung or failed to complete tasks a dozen times in the last week due to inference capacity.

Reply on news.ycombinator.com