Hacker News Reader

Yet, they're all almost exactly the same in style and composition.

Certainly not because all labs are pelicanmaxxing on the same exact image. Most patterns are similar everywhere. That's the result of training on the same datasets and scraped web contaminated with other models' outputs, combined with similar training methods and resulting mode collapse. Try if e.g. dolphins knitting socks in spacesuits are similar on all/most models.

This and your original comment aren't compatible.

How so?

> It's either a capability to generate arbitrary SVGs (or even something broader), or no specific training at all.

> That's the result of training on the same datasets and scraped web contaminated with other models' outputs

Maybe I'm misinterpreting though, or just semantics. I would consider contamination, or whatever the first example was that they all matched, as specific training data, along with all the duplications of. I doubt it's model collapse to the same pelican, without the same original example.

Perhaps you're right, this would probably take some rigorous forensics to resolve definitely. Somehow all models have very similar patterns in text, seemingly at random. I don't think everyone is trying to train on specific patterns on purpose, it's a pretty general phenomenon. (I'm talking about mode collapse, not model)

Reply on news.ycombinator.com