Is this essentially genetic algorithms for the LLM era?
They also often converge to a local minima, and are costly.
It’ll be interesting to see if LLMs change that, or whether we are just approximating something a gradient could do better
The innovation is novelty search, e.g. divergent rather than convergent optimization, paired with the capability that LLM genetic operators and representations unlock.
At least, that's my high-level take. google's alphaevolve is the same kind of thing but focused on convergent ga's (i think).