Not for me. It's turbocharged my career. It helps that I'm at the level where management isn't seeing my contribution as hours in a chair. Having kids gave me something in common with all of our senior management. Anecdotally, that improved relationship has been a hell of a boost for my career success.
It certainly will depend on your individual situation. If your management is all childless workaholics, I can see how it would totally count against you.