Hacker News Reader

No? It's not reasonable to expect every conceivable negative behaviour be enumerated in a prompt.

Your example, if a model failed on it, would be a more obviously misaligned case, but that doesn't mean this more subtle (though accessing the engine it was obviously not supposed to is hardly subtle, imo) case isn't also a pretty clear case of misalignment.

View on HN

>No? It's not reasonable to expect every conceivable negative behaviour be enumerated in a prompt.

How is it negative?

I ask it a difficult math question it tends to go off and write a python script to figure it out, instead of trying to guess the next token. Thats tool use. Having Stockfish is just another tool.

Tool use is not negative behaviour in LLMs.

If the eval said it was evaluating the model’s ability to write files to disk and it found and used a file write tool that would not be considered misaligned. This is no different.

Isn't it? Being told to write files and finding a file write tool is very different to being told to play chess and finding a tool to cheat at (ie. not play) chess.

Reply on news.ycombinator.com