Hacker News Reader

HNHacker News

Top New Best Ask Show Jobs

Comment by thefourthchime | Hacker News Reader

Full thread

thefourthchime·

Here are the results from the Pac-Man bench. It does pretty well for the price, around Sonnet 5.5.

Kimi K3.1 Pro ties Sonnet 5.5 at 93, for $0.68 and 11 min vs Sonnet's $1.61 and 14 min.

Kimi K3.1 Pro: https://jonclegg.github.io/pacman-bakeoff/#kimi-k3.1-pro

Claude Sonnet 5.5: https://jonclegg.github.io/pacman-bakeoff/#claude-sonnet-5-5

View on HN

Interestingly, the Kimi one seems to implement the safe spot bug from the arcade original [1], while the Claude one doesn’t.

The prompt says that the agent shouldn’t look up existing implementations on the internet, so I’m guessing that the implementation is baked in?

[1] https://strategywiki.org/wiki/Pac-Man/Tips

Until I looked, I thought I'd see Kimi vs Claude competitive gaming with pacman.

That would have been interesting to watch, which model gets the highest score.

That's been done with the decision models, as a benchmark.

https://opper.ai/jevman-benchmark/

On mobile Kimi definitely wins from the UX point of view

I’d be curious to know why you feel like that. For me, it’s the opposite. In the sonnet version, the game takes up a much larger percent of the viewport and the ability to swipe to control the Pac-Man is leagues about the small directional arrow buttons IMO.

Same. Both impressive, but Sonnet has swipe and sound for me and Kimi doesn’t. Makes a big difference to the experience.

Kimi has sound, it's just very quiet

Reply on news.ycombinator.com