There's an interesting parallel to be drawn here from prior RL research:
"Some evolutionary algorithms keep only the best performers in the population, on the assumption that progress moves endlessly forward. DGMs, however, keep them all, in case an innovation that initially fails actually holds the key to a later breakthrough when further tweaked. It’s a form of “open-ended exploration,” not closing any paths to progress. (DGMs do prioritize higher scorers when selecting progenitors.)"
Kenneth Stanley[0], the creator of the NEAT[1]/HyperNEAT (Picbreeder) algorithms wrote an entire book about open-ended exploration, "Why Greatness Cannot Be Planned: The Myth of the Objective".[0]: https://en.wikipedia.org/wiki/Kenneth_Stanley
[1]: https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_t...