Hacker News Reader

Comment by nicklecompte | Hacker News Reader

Taken literally this comment suggests that OpenAI et al actually want racist 4chan rants in ChatGPT's training data, or illegal child pornography in DALL-E's training data. Obviously that's not the case: the problem is that it's impossible to effectively audit the amount of data required to make generative AI work.

If Big Tech can't even catch pornography or explicit uses of the n-word, there's no way they'll be able to filter subtle LLM / art generator hallucinations.

View on HN

Going one level deeper, even AI-generated training data which doesn't contain hallucinations or other obvious errors could still negatively affect the performance of a new model. GenAI models have a tendency to repeat the same cliches, in sentence structure or the composition of images or whatever else, so even if their output is "fine" in the sense that it's factually correct and everyone has the right number of fingers, a model trained on it will still internalize those cliches and make them even more likely to show up again. That's even harder to audit out because it would only become apparent in aggregate when you realise your new model has learned to interpret "fantasy artwork" as "fantasy artwork in the generic Midjourney RLHF style" since it was trained on so many examples of that.

That's really just a feature of the default system prompt, although ubiquitous. You can character-prompt local LLMs as you like, but no one seems to do that.

ChatGPT doesn't take character prompts very well, that's why people don't know. Claude on the other hand is a champ with characters.

Not so much that, as OpenAI has finite resources and is allocating them to areas that are more valuable.

Has it been demonstrated that there is illegal CSAM in the DALL-E training?

I don't think it has, if only because OpenAI aren't transparent about where they get their image training data from, but the most widely used public image generation dataset was found to contain CSAM: https://www.theverge.com/2023/12/20/24009418/generative-ai-i...

Why would you need to?

Obviously generalizing over pictures of humans and pictures of porn will enable CSAM generation ...

Plus a LOT of things have been declared illegal pornographic material in various places: all porn (China), drawn CSAM, any kind of drawn porn with unclear ages, ...

> "Why would you need to?"

If someone is going to suggest that there is "illegal child pornography in DALL-E's training data" I think it needs to be backed up - that's a pretty big accusation.

It hasn't been demonstrated in DALL-E, but this was a problem in older OpenAI text-video models, and it's in plenty of CommonCrawl/etc data. There is anecdata from contractors suggesting that DALL-E generated CSAM imagery during RLHF, which is circumstantial evidence that similar images are somewhere in the training data. I think it's fair to say we can't rely on OpenAI's technical competence to remove this stuff from their training data, despite claims to the contrary w.r.t DALL-E 2. (Likewise it seems Copilot Designer churns out CSAM like candy.)

Your attitude is naive and competely at odds with (honest) AI safety research. The rational thing for users is to assume DALL-E 2 is contaminated by CSAM unless OpenAI can publicly demonstrate otherwise. Their leadership is too stupid and dishonest for people to take their word.

https://ieeexplore.ieee.org/abstract/document/9423393

https://arxiv.org/pdf/2110.01963.pdf?trk=public_post_comment...

https://www.theguardian.com/technology/2023/aug/02/ai-chatbo...

You're saying my attitude is "naive". I'm saying that it's irresponsible for you to state that there is CSAM in DALL-E's training data unless you can back that up.

I find OpenAI's refusal to document what's in the training data infuriating, but I that doesn't lead me to assume they weren't able to filter out CSAM from it.

If they don't care about this issue at all, why are they exposing low paid workers in Kenya to these conditions?

Reply on news.ycombinator.com