I think you misunderstand what the danger is. You cannot create
any safeguards that prevent an LLM from discovering exploits in software. Exploits and undefined behavior are inherent to the process of using a programming language; it's not possible to debug or read source code without also auditing it for exploits and unexpected behavior.
The danger is software vulnerability. When asymmetric war breaks out between the US and China, it won't matter who signed what. Both sides will immediately use uncensored AI models to attack civilian infrastructure (not just government assets), and whichever side has the more-vulnerable internet will lose. You cannot bolster America's cyber defenses while depriving consumers of capable cybersecurity models. What OpenAI and Anthropic are lobbying for is the right to tax US businesses for having poor security. China's models threaten their ability to profit off of the scaremongering that they market. It is dangerous for American privacy and security, but great for OpenAI and Anthropic's B2B contracts.