If you need to work more hours because your start-up is lazy about tracking productivity and "time in seat" == "more productivity" then try and build and document an empirical way to measure your productivity. There are a ton of good books out there on this subject. I recommend "Accelerate."
Go ask your grandfather if he wishes he had worked more or spent more time with family. Relationships NEED nurturing. Not making time to be together can and will kill your relationship.
Check out "Deep Work - Cal Newport" and "It Doesn't Have to Be Crazy at Work - by Jason Fried, David Heinemeier Hansson". You do read and do research on how to run your company and track your productivity, don't you? Or do you just fart your way through life and are consistently amazed at how the harder you work the further away the goal-post is?