Older versions of git would tell you the hashes were wrong. Implementations of git in other languages would tell you the hashes were wrong. Manually checking would tell you the hashes were wrong. It's just not feasible.
EDIT: Plus, git is stored in git, so you'd need to backdoor git first... ;)