This "from time to time" should be daily. A very big part of happiness comes from relationships. Often you don't realize this and even make jokes like this one:

Q: "Do you have time for private life in Silicon Valley?" A: "What's this?" and then LOL because you know you are doing too little for your personal life...

If you would give only 15% more of your focused time to your spouse, kids, father, mother, friends... they would value it a lot! And it would be your long-term success! Are you ready to make such a small sacrifice for your "private life start-up"?