First of all, and most importantly, most people do not give very much of their income away. And as you accumulate more money, your rate of giving tends to decrease. There are counter examples, thankfully (!), but this tends to be the trend.
There are also very important questions about the counter force of the labor you engage in to earn money. "Wall Street" type jobs make a great example here: say you earn $1M in a year by, say, funding expansive oil palm plantation development in southeast asian peat swamps. You may end up giving away a generous (!) 10% of your after-tax earnings, but you probably won't make up for the negative impact that you, or your money, had.
This particular externality is huge when you think about how investment works. For instance, all the major charitable foundations have large endowments that are tied up in various capital markets. Almost all of these foundations only give away their legally-mandated 5% payout amount. All the while, their complex financial machinery is generating, on average, returns in excess, or close to, 5%. So 95% of their money is actually working to make more money through the whole expansive and environmentally destructive capitalism thing[1].
Working at a soup kitchen full-time when you posses important skills does seem silly, though[2]. Like everything, I think there's a balance here. For instance, if you're highly skilled, donating a percentage of your time to volunteer or work at a lower-than-market-or-just-market salary at a nonprofit you believe in can have a tremendously important impact.
1. Not all foundations do this. A notable exception is the Bill & Melinda Gates Foundation, which I believe has a long-term goal of giving away all of their assets in some finite time period. A mixture of private investment in areas of interest can also help, e.g. what Omidyar Network does a lot of (http://www.insidephilanthropy.com/tech-philanthropy/2014/3/2...)