Me, I'm mid career, and wish I hadn't focused so much on doing more programming in my off time. Most of the side projects I worked on never really came to any sort of long-term fruition, and I'm not even sure that they were that helpful in my career. What really helped was being able to demonstrate domain expertise, and I built that up by becoming less focused on programming, and spending more of my work time just talking to people and understanding their business.
There are two big problems with just working on raw programming skills: First, if all you are is a programmer, then you're fungible. The only programming skills that aren't particularly fungible are skills in entrenched-but-uncool languages like COBOL and RPG. The cools ones are cheap; anyone who can stick to a MOOC can build up a portfolio of side projects on popular technologies. On the other hand, domain expertise is very difficult to replace. The resources to build it up typically aren't publicly available, so they're less common.
And there are plenty of side projects you can work on that give your brain a break, or at least a change of pace, while still looking good on your CV and opening up job opportunities you might not otherwise have. Lately, for me, it's been learning languages. It's a lot easier to fit into a life, because you can do it while commuting or doing housework. Having a business proficiency can definitely open doors. And, once you get past the beginner stage (which is best undertaken as a solitary pursuit), it's a _great_ social activity.