Your life will be more challenging if you start a SAAS business. It will not be easier. The only people in SAAS I know with an easy life poured their life into their company and things came good. Investors are starting to see SAAS as a 10yr investment, to give you a sense of time. Ironically those I know who have done well are some of the busiest people I know. I guess they are driven and love their work.
I think what you are describing is working part time. Maybe this is a valid option if you want to have more time doing your own thing?
The startup world likes the approach of make a plan and work backwards which is what you are currently doing. I would just suggest you re-frame your expectations. Read Peter Thiel's books but talk to the average SAAS Joe as well as we are not all SAAS rock stars :-)