Measuring "cost per successful PR" chose Luna 6 low as the best model

stet.sh

1 pointbisonbear2 comments

I haven’t really gotten into comparing models but I wonder if I’m doing something wrong or different, because results like these don’t match my experience.

I’m using OpenCode Go/Zen with OMP and generally DeepSeek Flash will complete a task for 10 cents or less.

Every time I’ve used any OpenAI models, even Luna, it burned through dollars for similar complexity tasks.

Is it the effort settings? I leave OMP on auto.

Something else?

effort makes a huge difference in cost/quality. in my measurements (https://www.stet.sh/leaderboard). Luna can definitely tend to overthink on the higher reasoning effort settings. I've actually noticed that deepseek 4.1 flash max (albeit the one before 10/17) tracks close to GPT 6 Luna max in price.

also worth noting that Luna is much more attractive on the Codex subscription plans due to heavy subsidization there - if you're using Opencode, I feel like Deepseke likely makes more sense, since they serve it very very cheaply ($60 or so of usage for $10).

Reply on news.ycombinator.com