Ideally, I'd do just this. I'd leave my job, take a few months and do my own thing. The problem is that I'm not prepared to take that risk; I have a good job, it pays well, and it supports my family. If I walk away and succeed (where success >= the same amount I'm making now), awesome. If I fail, however, I've left a job I'm good at, and that I'm comfortable at.
Perhaps that's the crux of the issue right there. I don't want to take the risk of putting all my eggs in one basket, but I find it difficult to get inspired if I'm not fully invested. Thus, if given the option after a long day of work to either watch Top Chef with my wife or do something interesting in Python, Top Chef+Wife wins every time.