I know some comic artists with day jobs, and pretty much every one of them gets up way early and gets drawing before going off to work. If they try to leave it until later life just butts in and it never happens.
Part of this is realizing that you'll have to do without certain things. You will not be able to watch every movie, every TV show, every spots event, etc. you might like to because there just isn't time. (And if not entertainment, then other things.)
It's a sacrifice, it's work, it's hard. If you can, try to arrange your project so that you hit useful, appreciable milestones fairly often. Divide your goals into small, manageable tasks. If you can see progress you'll find more motivation to continue.
And give regular updates here on HN so we can cheer you on.