(I'm assuming here we're not considering inflation. )
If you need it in 10 years, you're going to need an annualized 25% or so rate of return. That's unlikely.
If you need it in 25 years, you can get by with 10%. That's definitely possible, if you follow good principles.
On the other hand, if you can put in an extra 50k a year, You should have close to $1mm in 10 years. (550-600k in investment, 400k in growth).
Given that you hate people and are super smart, and have a self described penchant for gambling, I would suggest you not try to grow your business, and also that you stay away from the stock market as long as you think timing it is a great plan.
All that adds up to (in my mind) buying some real estate. You live in a beautiful vacation spot, and could find property managers so that you don't have to deal with tenants.
Your 100k could easily buy a 300k small property, with some left over for unforeseen problems. Try it! It won't fluctuate in value like your previous stock market purchases, that's for sure.