Hacker News Reader

Comment by ritz_labringue | Hacker News Reader

We've moved past per token cost a long time ago. Smaller models use more tokens. What's interesting is the cost per task, on that front deepseek isn't particularly good.

https://artificialanalysis.ai/#intelligence-efficiency-tabs

View on HN

The AA benchmarks seem to be mostly "complete this small intelligence-proving task from scratch". That consistently puts models with high intelligence to output tokens on the frontier, especially the OpenAI models, on the frontier. But a lot of software engineering work is many-turn work on established code bases. There's a lot of reading and a relatively large context unless it's carefully controlled. Writing only a few output tokens doesn't really cut costs because most of the spend is read tokens that get recycled through dozens of turns. That's basically the ideal case for DeepSeek because of how cheap cached tokens are.

It highly depends on the task. So far, on the tasks I personally benchmark, DS v4.1 Flash @ deepinfra.com is always cheaper because their cached input price for it is so very low and their cache hit rate is so very high.

do you generally favor deepinfra or you are mentioning them because it's 50% off on openrouter?

I really want to see some comparison on deepseek 4.1 flash 1017 providers, like, what's more trustworthy to not quantize without disclosing it, etc

also I can't find the official DeepSeek provider on openrouter https://openrouter.ai/deepseek/deepseek-v4.1-flash-1017

I use deepinfra directly. They have had a deep discount listed on their site for as long as I've been using them for that model and for v4.0 before that. I've used the DS v4.x Flash model from a number of provides and the inference quality at deepinfra seems to be as good as anyones.

That analysis is unfortunately running DeepSeek on Max. DeepSeek recommends against that since it guarantees higher cost and rarely improves the outcome- it should be ran on "high".

where does deepseek say they recommend against high?

They have "high" on there now too, just search for it.

Found it on that page, but they don't have it on their coding page.

Their coding page shows DeepSeek Max as Pareto optimal: https://artificialanalysis.ai/agents/coding-agents Costing 1/2 of Terra Xhigh. It would also be great to see Sol and Opus not on max there.

Not a useful benchmarking site for coding unless you only want to run your models on Max.

This benchmark is total bs and does not represent real world usage. None of the models that “beat” Deepseek in the benchmark are any good.

Reply on news.ycombinator.com