I got a question after checking results on the open LLM leaderboard[1].
Comparing the result of NyxKrage/Microsoft_Phi-4 and microsoft/phi-4 or unsloth/phi-4, I can see fixing both the tokenizer and chat template causes the performance of both IFEval and BBH to increase. However, the performance on MATH, GPQA and MUSR degrades A LOT.
Is there any explanation on why this is happening?
[1] https://huggingface.co/spaces/open-llm-leaderboard/open_llm_...