1. I don't think there are any part-time CS PhD programs, so I'd have to quit my full-time job.
2. My full-time job brings in 3/4 of the income and my wife isn't in a position to earn much more than what she's earning now, so we'd have to suffer a big drop in income.
3. The big drop in income (even if I get a TA or RA position) would force us to move from our home to either university or an insufficiently-sized (that is, insufficient for us + the children we plan on having in the next ~3 years) dwelling.
4. My wife has no interest in earning her own PhD and doesn't want to work while raising our children. Furthermore, I don't want her to work while raising our children.
I don't think it's possible to do without being a selfish arse.