Speaking from experience, I have often taken the hard path. I've always regretted it and often, if I'm being honest, it's out of a misguided sense of stubbornness.
That's why I recommend trying college, to the OP. Maybe he can learn advanced mathematics by himself, in isolation. But that isn't the maximum-likelihood path. And when you're doing something hard, I've learned it pays to do everything you can to stack the deck in your favor.
I think the main thing is to realize you only live one. I have friends who are more successful than me in every way. One earns literally 10x what I do, others work at more famous companies, still others receive absurd amounts of money from their parents for sitting on their ass. You can't choose your hand in life, but you can choose how you play it. So it's really up to you, whether you want to play the game on easy mode, or hard mode. Or whether you want to feel sorry for yourself or just make something of the hand you're dealt, though it may not be "fair".