The Poison of (LLMs) Alignment
arxiv.org
arxiv.org
I did not say they asked the AI to lie. I did they asked it not to give a correct and truthful response.
Maybe this explains much of the deterioration of ChatGPT that has been reported.
The largest Llama2 model has 70B parameters. They ran the experiment with 7B Llama2.
Likewise, any pre-prompt or fine tuning training that is NOT relevant to the subsequent prompts adds unhelpful and highly non-contextual complexity to the models job, so likely to reduce the quality of response.