This is a great idea, but I'm thinking weights aren't enough. If the recent rumors are true of the latest models like chat GPT-4, for example - they have a realitively new style of overlapping _a cohort_ of models, not just utilizing a giant matrix of weights (as has hereforto been the defacto architecture) - or is this still the case even for chat gpt 4 after training?

I'm no expert in ML, so perhaps I am ignorant as to what information is truly needed to reproduce results. I just cant help but think you'd need some hints at the model architecture itself as well.