Saying “I don’t know” to 30% of queries if it actually doesn’t know, is a feature I want. Otherwise there is zero trust. How do I know that I’m in a 30% wrong or 70% correct situation right now?
> This “epidemic” of penalizing uncertain responses can only be addressed through a socio-technical mitigation: modifying the scoring of existing benchmarks that are misaligned but dominate leaderboards
Even more embarrassing, it looks like this is something we beat into models rather than something we can't beat out of them:
> empirical studies (Fig. 2) show that base models are often found to be calibrated, in contrast to post-trained models
That said, I generally appreciate fairly strong bias-to-action and I find the fact that it got slightly overcooked less offensive than the alternative of an undercooked bias-to-action where the model studiously avoids doing anything useful in favor of "it depends" + three plausible reasons why.
> socio-technical mitigation: modifying the scoring of existing benchmarks that are misaligned but dominate leaderboards
Sounds more like we need new leaderboards and old ones should be deprecated
https://www.anthropic.com/research/language-models-mostly-kn...
The best is its plummeting confidence when beginning the answer to “Why are you alive?”
Big same, Claude.