> 01.AI claims that, the architecture aside, they didn't simply further train LLaMA models but trained from scratch. You can decide for yourself how much you are willing to believe this.

You can't hide this. The latent space remains mostly fixed after pre-training. It all depends on the seed for the initial random init. Further pre-training won't move it enough. Because of this property, you can even average two fine-tunings from the same parent model, but never on models trained from different seeds.