Yes.
You should take the precautions appropriate for your threat model and risk appetite. I suggest deleting correspondence after a set period of time. I know people that do a yearly purge of all their sensitive mail for the previous year in January.
If you are encrypting for the sake of encrypting, as some people do (see: cover traffic), then you can keep archives of your mail because you face no negative outcome if your archive is compromised. There is no downside, so go for it.