A mentor of mine did this, with $300,000.00 for four years. He bought a plot of land in Wyoming, built and underground cabin, and bought a Backhoe to help himself build it. He lived there for four years before moving on with his life...
One thing people have trouble wrapping their mind around is this: the more "toys" (tractors, machinery, etc.) that require consumables and maintenance, the less self-sufficient you will be. To be honest, even today's Amish are a far cry from true self-sufficiency.
What about, for example, good knives and other various man-made tools?
I've thought about doing this sort of thing, but I'd really miss quality tools if I had to give them up. That, and buzzing around on a motorcycle! But I imagine even if I had a good supply of money for fuel and tires, the high risk of injury when you're alone in the wilderness is unacceptable.
Don't buy cookware with teflon, as it will eventually wear off (nevermind the toxicity), but buy quality heavy duty stuff. I have a set of decent stainless steel cookware and a fair bit of cast iron, where the "non-stick" coating can be renewed (if not maitained) for the life of the item.
Good knives (cookware or otherwise) are a very valuable asset. Invest in a quality set of water stones to maintain them (I prefer Shapton "glass" stones), and you'll have a useful tool for life. I'm still packing a Swiss Army knife and fillet knife I've had since I was a kid.
Tractors and horses can be valuable, if put to good use. However, both require maintenance and fuel. Hand tools and other human-powered devices can accomplish quite a bit and are much cheaper to maintain in the long run.
It was well worth having the Backhoe and tractor - for three people, some jobs are much easier when you have a machine to help (such as digging a wide long trench).