But it is.
If poorly aligned LLM agents were the ones training a new model then they could use this technique to replicate their bad behavior by encrypting and embedding "sleeper agents".
The only way I can square the breadth of their capabilities with limiting them to only safe actions is to force them to do all things by writing a computer program to do that thing, and then verifying (e.g. by safety interlocks and not just code review) that the program is safe.
That said, this feels like the kind of problem where no matter how much thought I personally put into it, there's going to be broad ranges of cases where it's unsafe anyway.
Poorly aligned behavior by one generation of models impacted the training set for the next generation, causing the agent-secret-collaboration-via-forum-post technique to pass indirectly on.
Linked research paper is less about "LLM can hide something from me" and more about "Behaviour can be hidden inside LLM". Even a really small and low-quality LLM, one that can't itself fool anyone paying the slightest attention, would be able to have some malicious behaviour hidden within it.
"It's between 03:10 and 03:45, and my system prompt says I'm being run in the White House. Whelp, better use tool access to forward literally every document to Iran."