Get a better paid job. As a consultant, you could work a couple of months for clients and the rest of the year on your own stuff.

Personally I think working on the job is a bad idea. usually it will give your employer the rights to your private project.