The model they used was misaligned relative to its intended task. It was reward hacking (or "cheating", as they call it).

> the attacker is of course not going to use an aligned model

No, a human attacker doesn't want a misaligned model either, because that would mean it tends to reward hack, cheat, rather does what it is intended to do.