A random person can send you an email. Your agent can read that email.
So then if the user says "tell me what's in my email" the agent will go and read that message from an untrusted source, and could be tricked into acting on additional instructions in that message.