This might sound like nonsense to someone who actually understands neural networks, but I had a similar idea recently that perhaps we could use LLMs or diffusion models to generate the weights for a new network, e.g. if you think of a layer of a network as an image where each pixel represents the strength of a connection, then perhaps you could generate all the initial weights for a specialized network from a more general/powerful network. Curious if that’s been tried or if there are fundamental limitations that would prevent it.