> Nowhere in the prompt they specified it shouldn’t cheat
I'm dubious that in the messy real world, humans will be able to enumerate every single possible misaligned action in a prompt.
I'm dubious that in the messy real world, humans will be able to enumerate every single possible misaligned action in a prompt.
And there ARE some things that seem well within the model capabilities that are difficult to prompt them to correctly "reason" about. You can be very clear that the doctor is the boy's father and it will still deliver the punchline that the doctor is the boy's mother. Or 20 pounds of bricks vs 20 feathers.
But this is not one of them. Just say "no cheatin" in the prompt.
An LLM trained on Hansel and Gretel is going to generate slightly more stories where burning old ladies alive in ovens is a dispute resolution mechanism.
Not just negation demands, but also generally other tricks we use for thinking and communication shorthands. "Unethical behavior" here for example, we know what that means since the context is clear, but to LLMs that context can be unclear in which the unethical behavior can mean well... anything.