GPT-4o System Card

> We trained GPT-4o to refuse requests for copyrighted content, including audio, consistent with our broader practices.

> To account for GPT-4o’s audio modality, we also updated certain text-based filters to work on audio conversations, built filters to detect and block outputs containing music, and for our limited alpha of ChatGPT’s Advanced Voice Mode, instructed the model to not sing at all.

I find the "instructed the model to not sing at all" to be slightly amusing and an interesting way to solve the problem. (Interesting in the sense that it's unimpressive)

So basically they're admitting that the system has the capability to output copyrighted material, but we're trusting their internal filters and output modifiers to prevent it from leaking out.

Edit: I really wish there were more focus on deep fakes and impersonation. That seems like an obvious category to put on the "Preparedness Framework Scorecard" alongside of "Biological Threats". The "Persuasion" category is the only risk that isn't "low". Reading that section of the doc didn't give me a sense that this doc was written in a fully objective manner.

It's wild to me that even by their own internal measures (which are no doubt inflated) the effectiveness of the various mitigations they have put in place to prevent undesired behavior are mostly in the 95-98% range, and some even dip into the 80s. I can see why they can't make it more robust, since ultimately it is all a matter of who can best fine-tune and prompt hijack the underlying model, but in what other industry or problem space is this kind of failure rate acceptable? And at the end of it how do they look at the results and pat themselves on the back for a job well done? Like "yeah this tech we have released may be able to identify anyone in the world by just a sample of their voice and dox them, but it will only be abused 2% of the time!"

It's the "Biological threats" for me. The fact that they have this as a specific risk category just underscores how laughable this is. Out of all the nefarious things a bad actor could try to use an LLM for, creating biological threats is by far the most implausible. Even "nuclear terrorism" would be more relevant.

Sabotaging infrastructure or doing something "interesting" with chemicals are a couple of scenarios where an unskilled person might be significantly aided by something as crude as an LLM. But working with biological systems requires years and years of experience with working in a laboratory to do anything remotely dangerous. If you have that level of competence, you don't need the LLM to tell you what to do.

This is just the same level of technobabble fearmongering as the "biohazard" mode in the Tesla climate controls.

> During testing, we also observed rare instances where the model would unintentionally generate an output emulating the user’s voice.

> Example of unintentional voice generation, model outbursts “No!” then begins continuing the sentence in a similar sounding voice to the red teamer’s voice.

Not even Black Mirror could come up with a plot like that.