It used to be that religion provided these answers. Believe in God, have faith, go through these rituals, help the poor and you're a good person who has successfully discharged moral responsibility.
Religion has faded away, so we're left with "a good person helps the poor" which everyone agrees is a pretty moral thing to do. But the more indirect it gets the less moral value is assigned to it.
People assign moral value to hours of time invested because time is what we all have equally. Time is not money when it comes to judging character. It must be this way because otherwise it'd be that very high earning people help the poor to a vastly greater extent than the average person does, just by turning up to work every day and paying their taxes. If that automatically made them morally superior to everyone else, it'd create a lot of resentment. No matter how hard you worked you would never be as "good" as the rich guy who just turns up and does some lucky stock trading or whatever.
I think sometimes programmers don't feel this way to quite the same extent as other sorts of office workers because we sometimes get more meaning from our work than someone in e.g. sales. The Silicon Valley stereotype of "Join our startup because we're changing the world for good!" has a basis in real self-perceptions. If you believe you're making the world a better place by actually doing your work, of course doing extra charity stuff will seem silly and a poor allocation of time/money relative to other ways of achieving the same ends.