I'm too lazy to look it up, but someone on HN linked a drone test done with an AI pilot that was tasked with destroying surface-to-air missile targets (in a simulation). At one point, the human operator instructed it not to hit certain SAMs, and since the goal was to destroy SAMs, the AI took out the base with the human operator.

On the next run, they instructed it not to take out the human operator in pursuit of its goal, so instead it targeted the radio towers the human used to instruct it.