Hacker News Reader

There's also a tokenizer efficiency difference: modern Claude's 100K tokens are about ~60-65K modern GPT tokens, so in reality the Luna cutoff is much further away than the Haiku one.

You can test with Anthropic's count_tokens endpoint or with https://crates.io/crates/tokwc

View on HN

> ...this tokenizer, the same input text produces approximately 30% more tokens on Claude Haiku 5.5 than on Claude Haiku 4.5.

So, it is might be even worse.

No, it's just Haiku 4.5 is so old that it predates the new Claude tokenizer change in Claude 4.7+

Reply on news.ycombinator.com