Edit: I'm not 100% sure, but I think my comment is being interpreted uncharitably. I would like to grab your finger, which is pressed squarely atop "output of other LLMs", and move it over a bit to the label of "website with unknown and almost certainly copyright content". And then, I would like to point out that there is a difference between training on such data with the benefit of plausible deniability (e.g. some third party), versus going specifically after data you know to be copyrighted, and then claiming that this is precisely the reason why your model will spit such content out verbatim. At that point, you might as well indict yourself, no?