After a while most people will realize that your spouse/kids would rather need money not time from you. Its far better to have a rich dad/husband than poor one who spends more time.
Over time even the guy who made this choices will know that, every time he sees someone buying a new car, going on a costly vacation, sending kids to a Ivy league and all the big money luxuries. Inevitable peer comparisons happen, and things get out of hand pretty quickly.
You generally get to hear 'How I learned to be happy with little' kind of stories from such people.