Hacker News Reader

Fundamentally untrue, and disheartening that it's the top comment.

You can't use a model's output to train another model, it leads to complete gibberish (termed "model collapse"). https://arxiv.org/abs/2305.17493v2

And the Llama 2 license allows users to train derivative models, which is what people really care about. https://github.com/facebookresearch/llama/blob/main/LICENSE

View on HN

The truth is between these two. You can use a model’s output to train another model, but it has drawbacks, including model collapse.

Some of the best LLaMA finetunes today are trained on GPT-4 output.

Yes, you cannot do this kind of thing indefinitely and expect endless improvements from "endless training set". But that's a very different problem.

Reply on news.ycombinator.com