Quick reply to my own comment. But another reason it seems this is helpful is that in many reinforcement learning tasks, like games, rewards are few and far between, so these goals also give the agent oncentive to keep trying new things / learning about the environment / task in the presence of sparse rewards?