I guess one technique is to train a model on various language model outputs to classify these as good/bad/intriguing/robot-sounding/repetitive/etc. A human can tag the answers for the training dataset.
Then, we can use this model to compare different LLMs and optimize new models - could be with genetic algorithms or just a human tweaking the model - so that the LLM maximizes whatever we want.