(foreword, I'm far from well versed in Japanese society)
From what I understand, traditional Japanese family model is a stay at home mother with an almost stay at work father.
Society is organised around this model, e.g. no day care, mothers not expected to build a career, and probably being a father comes with pressure about being a reliable provider (getting overworked). I remember that a lot of father losing their work preferred abandoning their family than enduring the shame of losing their job during the late nineties asian crisis.
Regardless of immediate material complications of begin parent, the additional social burden certainly not help.