As not all people should be musicians professionally, not all people should be programmers professionally. It's hard to put my finger on it, but in both cases some people just don't 'get it'.
Of course, if you are talking about having fun on your own - writing apps or making music - or 'jamming' code or music with your friends - you can do whatever you like.