In the dark times before PyTorch there was an idea called NEAT: "neuroevolution of augmenting topologies", which tried to use a genetic algorithm (i.e. testing a bunch of slightly modified solutions for loss) to discover both optimal weights and optimal network structure. I don't think this gets used all that often, if at all.
I hear stories every few years about how many models have unused neurons that can be pruned or how hyperparameter selection is a pain in the ass, but nothing about automating the pruning and parameter selection in such a way as to efficiently use the whole model. I'm not sure it's necessary anyway.