Why do you think it’s misleading?
You think it’s just generating plausible random crap that happens to exist verbatim on the internet?
I mean… read the paper, 0.8% outputs were verbatim for gpt-3.5.
I’m not sure how you can plausibly claim that’s random chance.
> I think it’s a bug
It is a bug, but that doesn’t make it misleading or untrue.
This is like saying a security vuln in gmail that lets you steal 1% of mail is misleading. That would not be a bug, it would be a freaking disaster.
The problem here is that (as mentioned in other comments), training LLMs in a way that avoids this is actually pretty hard to do.
/shrug