HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by ncruces | Hacker News Reader
Parent
Full thread
ncruces
·
Did they ask Claude "don't cheat" this time round?
View on HN
ModernMech
·
You joke but when benchmarking with ai this is important. When you tell it optimize X, it will do so at any cost, including cheezing benchmarks. So you have to make sure you tell it what it can’t optimize along with what it can.
Reply on news.ycombinator.com