If he is not living with the kids, then his wife does all the kids care? and pays rent for the kids?
Also, why would the kids and wife want to go and live "nowhere" when he builds his dream house, don't they live an ok life where they are and don't need to relocate to a lifestyle that's not theirs?