This is my experience as an adult studying calculus, linear algebra, dynamic programming etc.

Calculus is mostly algebra. If you can’t reliably solve algebraic equations, derivatives and integrals will probably be a step too far.

Algebra is mostly arithmetic. If you can’t reliably add and multiply rational numbers, solving for unknowns is going to be patchy at best.

The key part as a student is learning to recognise when I needed to re-tread the primitive operations. Having to go back and practice the mental arithmetic I first learned in primary school was a hit to the ego, but at least I could recognise and take that step. When I was in my late teens / early 20s I was much less able to swallow my pride and back-track.