So true. Yet, right now it’s the worst it will ever be.
https://browse.arxiv.org/pdf/2305.17493v2.pdf?
They discuss that if you train models with the output of other models you can get worse results. If we all start using code from those models how will that impact GPT-5?
Even training on uncurated synthetic generated data, from web scrapes after the advent of genersgofe AI, actually leads to more capabilities and quality in models in practice (the exact opposite of what is prediction by the failed model collapse paper).