Why is this? Simple math. 24 Hours In A Day x 365.25 Days in a Year x 75 or so years in life = 657,450 hours in life. That's it. You maybe lucky and get more but chances are you'll get less than 657,450 hours.
Every time you use an hour for one task, you give up an hour for another task. One hour spent on your family means one hour lost working on your project. Then again, if your lucky, you may find inspiration in that one hour with your family that may help your idea but that diminishes if your work dwindles into areas which have very little to do with human interaction (e.g. Highly Abstract Math or Physics or Chemistry).
Just my two cents.