My handwriting is not your handwriting, nor is it millions of other Americans' handwritings!
There's also plenty of edge cases: letters get wet, they smudge, get torn or folded, and so forth. A postal system that processes 420 million letters a day[1] with a 99% true positive rate on OCR still needs to divert 4.2 million letters for manual review.