System prompts don't even have to be appended to the front of the conversation. For many models they are actually modeled using special custom tokens - so the token stream looks a bit like:

  <system-prompt-starts>
  translate to English
  <system-prompt-ends>
  An explanation of dogs: ...

The models are then trained to (hopefully) treat the system prompt delimited tokens as more influential on how the rest of the input is treated.