Keep them away from both the image and the container! Getting env var values dumped for a process is trivial outside of the process and even easier within the container process space.
common info pages (ex: phpinfo), core dumps, debug errors and logs are notorious for containing them. And those aren't even counting the ways a malicious actor can persuade a program to provide them.