Math is certainly not for everyone, but I actually found that once I started diving into pure math (NOT what was taught in my school's math curriculum) it was a whole new world.
Did your school actually teach pure math?
I bet a lot of programmer types had similar frustrations to the computationally intensive version of math taught in many high schools when they knew that these processes could simply be automated.