It feels weird to me to use the hyper parameters as the variables to iterate on, and also wasteful. Surely there must be a family of models that give fractal like behaviour ?
If you envision a given architecture as a class of (higher-order) function, the inputs would be the parameters, and the constants would be the hyperparameters. Varying the constants moves to a different function in the class, or, varying the hyperparameters gives a different model with the same architecture (even with the same data).
Yes, I also think this is strange. In regular fractals the x and y coordinates have the same units (roughly speaking), but here this is not the case, so I wonder how they determine the relative scale.