I'm not an expert, but I wonder if the wide network could then be trimmed down for special purposes? E.g., find the parts of the network needed for a certain task by looking at activations on inputs and dump the rest.
Besides pruning (which is what you describe) you might find the "lottery ticket hypothesis" interesting. The idea is essentially that in a randomly initialized large network there is a subset of weights that are already closeish for a given task and training helps tune that and suppress everything else, and that knowing this you can actually get better results by pruning before training.