I think it has much to do with the abstractness of mathematics (especially category theory). Some just have a hard time anchoring the ideas to something concrete, and they are in need of that anchoring to make the ideas stick. With engineering and physics equations there is always something physical to relate them to (at least somewhere in the universe(s)) -- not to say there aren't harder equations to grasp in those fields too, but again that is typically trending into the more abstract (less macro) areas of the field. The trouble with math is sometimes you just have to work through equations enough until you form some intuition about how they work. Some folks just want to skip that step and others Oblidge them by assisting their mental model with metaphors which seem to never be universal enough -- if it were it would just be math.