llms are in fact, not better than traditional chess engines at chess. most benchmarks that llms are trained for are coding, math and science benchmarks, that's why performance tends to be spikey