Speaking from my own experience, I'd say that my biggest hurdle to following my dreams is a false dichotomy created somewhere in my imagination. For some reason, I always imagine 1) no income, 2) no reduction in expenses, and 3) not being able to get a job should I go belly up.

I started working part-time about 5 weeks ago, and it's become obvious that I was fooling myself for the sake of maintaining the status quo. Now that I see that it's relatively easy to find income outside of my day job and that my family can cut expenses if we are more deliberate about how we spend, I wonder why I have always imagined the worst.

The biggest factor is stuff. It's true that what you buy enslaves you. Houses to a mortgage, cars to a loan and expensive insurance, retail items to credit card debt. The more you can eliminate or reduce those expenses, the less of an excuse you'll have to keep working in a situation that isn't fulfilling.