No. No no no. Yes, arithmetic math does benefit from repetition and memorization, is useful, and your comments on why getting good at arithmetic is hard are spot on.

But math is so much more than arithmetic. Math is about learning to apply a variety of available solutions and processes to solve a problem. Math is about understanding relationships. It's why I get better at math in a combination of my physics and programming courses than I ever did in algebra or calculus.

In fact, the biggest problem with mathematics education is that it is not falsifiable. Measuring reasoning, logic, critical thinking, and problem solving skills is almost as hard as teaching them. Writing a generally applicable test that measures whether a teacher taught you to actually use and solve problems with math is nigh-impossible, made worse that students (and most of all, parents) don't want to be held accountable for problem solving, because it's too fuzzy, or because you just "have to be smart". The fact that real math skills are tough to teach and tough to measure is the problem, not the reverse.