I honestly never got why the belief that if you used this line of reasoning in practice, you were just a bad guy, that thought they were a good one, never seems to be a more common belief
I mean it pops up in media a lot, either as misguided, or deceptive characters
Take the US recent assassinations of Iranian leadership. Objectively unilaterally assassinating other countries leaders is wrong - but somehow it's fine if you have a good guy/bad guy lense.
One of the problems here is huge swathes of material that the AI will have ingested includes this kind of exceptionalism thinking - it's particularly deeply rooted in Western culture generally and US culture in particular.
The problem isn't so much whether the good guy/bad guy allocation is corrrect - rather that it's very existence in the training set is deeply dangerous - as all it takes is one single classification to happen and suddenly everything is for the greater good.
Why? If the leader is responsible for atrocities, then why would it be wrong to kill them? Especially if their removal would result in a more liberal leader replacing them, and not just chaos or more bloodshed.
In the case of Iran, there was no evidence of that, but as a general rule leaders should not be a protected class.