Next time you're involved in a group project and end up doing all the work while the rest of your team receives the same grade, get back to us on what you just wrote.
Looks like you want to make your own, free will decision about what to do with your money. That's what private charities are for — that way, we both get to decide for ourselves, exercise our choice and you get to feel morally superior (and get the deserved praise), and someone else gets to use their money in other way.
When, however, state takes this money from you without giving you any choice over it, using it's monopoly on violence to force you into doing that, that's no longer your moral decision. That's just inevitability.