Let's get sci-fi and ponder.. weights programming replaces conventional training. Is something in the veins of the Ken Thompson Hack conceivable? A NN secretly injects its weights that "procreate"?
This takes that to the next level: while you're telling it to update a model's weights to improve math skills it might also tweak them to favour a particular RNG design weakness, a penchant for reducing the salt in cooking, and a preference for the early works of Huey Lewis and the News.
You can add layers to an existing model without massive training costs, which is of course obvious, but ideally you would adjust the weights of a number of layers. Ideally this would be done without knowing the weight decryption inputs, i.e. just a worm behaviour.
I recommend the tool hashdeep to compute and verify weight checksums trees, and signatures of those checksums with gpg or minisign [1], so you can always verify weights before they are loaded into your engine.