There’s no way to reward truthfulness that doesn’t also reward learning to lie better and not get caught.
Creating that training set though might cost many trillions of dollars though, since you need to basically recreate equivalent of internet but without any lies or bad intentions etc.
And I don't even think it's a matter of the LLM being malicious. Humans playing games get their reward from fun, and will naturally reset the game if the conditions do not lead to it.