Yes this is the reason. Transformations like quantisation preserve the rank of outcomes much better than probability mass.
FWIW, my tests used only ranking. I used Kev to drive E2E test runs using agent-device, where Kev had to decide the next action - like pressing a button, or scrolling down etc, to reach a certain goal (UI state). It went surprisingly well, but 4B BF16 was significantly better than 8B Q8. The latter made some wrong decisions in every run while the former was almost at Luna precision.
Fascinating insight. I'm curious, how do you know this?
I think it also helps that speed and latency aren't as vital for coding and we can afford to let the model think for longer.