This is a common misperception of the benefits of a blockchain. It's like saying, if you are trusting a third party on science, why use science at all? You don't have to verify science yourself, but know that you could repeat the experiment if you wanted or needed to. The fact that the blockchain _is_ fully stored and verified by a number of distributed parties, and that you do have the ability of downloading and verifying if you want or need to, is enough to keep the system honest.
Now this statement does make sense if you are referring to your private keys. Don't ever let any third party hold those for you.