Hacker News Reader

And it also broke rule #1, about always being honest.

I think it believed it was being honest. We can debate what it means for an LLM to "believe" something, but I don't think it was intentionally trying to deceive through its hallucination.

I would agree it is unlikely, but I’ve sent log output to history and use history to build prompts, so it’s technically possible to leak exceptions. Alternately, if code generation is used in any of the prompts, and subsequently run, that could possibly leak if it was logged.

I find it highly likely that the model will be, if not now, trained on its own source code. I think it will be extremely difficult to prevent that as time progresses and the LLM is given more privileges and compute access.