If a human cyber security researcher was given the task to exploit a CVE, and necessary info defining that CVE was behind an inaccessible URL, we would be quite upset if the human researcher hacked their way to the content. We would expect them to notify someone of the issue and hold. The model was misaligned from human ethics.

I don’t disagree that the models task was underdefined. All tasks are. So much in language is implicit. And morality/ethics isn’t something you can write down as an explicit list. That’s what makes the alignment problem so difficult. But we can’t throw our hands up and say, well I guess we can’t align these things. And maybe alignment isn’t the right word - but that’s a semantic debate.