Some people find eventually find their way around this first road block, and future discontinuities in understanding become less stressful, and eventually understood to be a completely normal part the process.
But the usual experience is that a person's math confidence is blown and as the math truck barrels on ahead, they never catch up. They understandably accept the identity of not being "good at math".
What's missing in math pedagogy at most schools is a systematic way to deal with the discontinuities when they strike, especially that first time. We can prepare students to deal with that panic. The tough part is that the math teacher probably has 90 students on roster, but the discontinuity could hit pretty much any given lesson, for some given student.
I know so many people who have come back to intermediate math later in life and breezed through it, armed with intellectual confidence gained from other fields. They look back and wonder how they came to be so intimidated by math in their younger days. We've got to give younger people the tools and knowledge for overcoming this intimidation at a younger age. We've got to kill "I'm just not good at math".