The tokenizer differences are major as LLMs are sensitive to whitespace handling. If I am reading the github page properly, OpenLLama failed to learn how to model code properly? Code contains many implicit reasoning tasks.

What other differences are there? The page doesn't mention how numbers are handled. These are two major things that impact model reasoning and numeric ability.