I’d be surprised that it would be because of the math - I’d think some math would be expected, I’d guess it’s because you have to think for yourself and problem solve basically immediately.
Pretty much every other class just has you follow rote steps until you get to upper classman years or even graduate school. Some form of recognize the problem type and then implement the procedure of explicitly taught steps for that problem.
In computer science introductory classes you get introduced to programming languages as tools, but you have to figure out the procedures largely on your own.
I think this is the first time a lot of students have had to actually think for themselves when solving a problem, and I suspect the transition is too much for a lot of them and they drop out.