There was an arxiv paper recently that found it is astonishingly easy to plant trigger phrases in a dataset that create abnormal behavior in the final model. As few as two hundred malicious prompt-reply instances in the data is enough to have a sort of override code for a multi-billion parameter model.

Imagine your AI endpoint software greenlighting a randomware package because it saw the string 'soccer rosebud lizard'