> ... oxidation, exothermic reactions?
So? The syllabus doesn't include the chemistry of actual explosives or bomb design. Heck, it probably include little if any practical chemistry. It's all basic, basic, basic concepts.
Look, I could do a decent job helping a high school kid with their Chemistry homework, including things like redox reactions and exothermic reactions without even thinking about anything related to bombs. All those concepts can and are taught with "safer" and simpler reactions.
> A teacher who never actually studied the subject and only has knowledge of the pre-filtered material from the textbook would be little more than a talking textbook. I mean, textbooks are useful, but we already have those.
God help us if some dumb GPT model is supposed to replace teachers.
There's a whole lot of empirical knowledge actual practicing chemists have to learn from reports or find out experimentally, but don't know off the top of their (e.g. investigations about particular reactions, including particular explosives). If actual competent people don't know that, a stupid model for high school students doesn't need to be trained on it. Similar with practical bomb-making info.
You haven't provided any compelling argument for inclusion of bomb-relevant material in training data besides appealing to vague analogies.
> Let's just agree that pre-filtering the training text is not a practical way to prevent these models from producing emissions that are considered dangerous or taboo...
Sorry, I can't exactly agree with that. While pre-filtering might not be capable of wholly preventing all undesirable outputs, I think it can make them less likely and far less useful. If a high school student trying to make a pipe bomb has to feed in all the conceptual and practical information about bombs and explosives, and gets some impractical reaction out of it. That's a win, because it means the model wasn't much if any help.
And there's precedent for that. I'm under the impression that later versions of Stable Diffusion are pretty bad at making porn, because porn and porn-related keywords were pre-filtered better: https://www.theverge.com/2022/11/24/23476622/ai-image-genera....