But from the paper, it sounds like this happens only during training. Some trick about constantly re-injecting the system prompt during chat conversations.

But during inference, there's no trick. The system message remains once at the top.