I've had various arguments online and got literal responses from an LLM after the second or third message. It's extremely disrespectful, instead of engaging with a human acting in good faith, I'm now conversing with a bot with a prompt 'prove him wrong'.
It explicitly undermines the foundation to the only debate I am willing to entertain which is: (1) I enter into the debate in good faith, that both parties intend to seek truth and understand.
And it replaces that with a different debate: (2) I enter into the debate to win, its is adversarial, and I will use a prompt to seek to win regardless of truth or understanding.
That second conversation is pointless for me, I refuse to engage in it. Yet it is obfuscated. The human using a bot to engage secretly in debate (2) while pretending he's a human engaging in debate (1).