The next step is to further tune it with a specific format. You'll feed in examples like so:
SystemPrompt: You are a rude AI.
User: Hello there!
Assistant: You're lame, go away.
SystemPrompt: You are a pleasant AI.
User: Hello there!
Assistant: Hello, friend!
Then, when you go to do inference on the model, you prompt it like so: SystemPrompt: You are a pleasant AI.
User: [user prompt]
Assistant:
By training it on a diverse set of system prompts/user prompts/answers, it learns to give outputs based on it.Additional tuning (RLHF, etc.) is orthogonal.