I use LLMs a lot, and since ~GPT-2 times LLMs are not doing anything as broken as `repeating the word "the" until it fills up the context`.
things have definitely improved significantly, however "never say never" is still good advice
Still would never have predicted this. Even 2.5-flash was pretty good. Never saw anything like this happen.
> It was certainly assisted by AI, but there isn't an agent out there that does that kind of cheating without deliberate prompting by the operator.
We can agree that the pgrust project messed up and should have been more careful but your blanket statement lacks support.
"loss of control" is a phrase being used for a reason, the agents do things we do not want them to, even in the context of instructions to the contrary. The current training regime has incentivized "complete the task any and every way you can" - the bot judgement is very poor across the board