Some would argue that the government is the biggest waste of money there is. I'm not saying that's my belief, but let's entertain that thought for a minute. If you believe that the government wastes most, say 95%, of its money on stupid things that should never receive a dollar from anyone (like the war in Iraq, to pick a random one), and you believe that you personally can do more with that money in terms of "making the world a better place", then it follows that paying as little tax as possible is the ethical choice.
Let's take an example: Bill Gates. Bill Gates is currently the second richest man on earth, according to http://en.wikipedia.org/wiki/Forbes_list_of_billionaires . Did he use all those tax schemes on his way there? You bet your bottom dollar he did. Now he's using this money directly to solve problems like curing Africa of Malaria. Do you think he's using that money more efficiently than a government programme would? I'd bet on that too.
Would the world be better off if the US government had $20b more and Bill Gates had $20b less? I don't think so. I think Bill Gates can use the money much more intelligently than the US government ever could.
So, is it unethical to pour less money into the pointless furnace of wasteful government?
I'm not saying this point of view is the only point of view out there. But to declare tax avoidance outright "unethical" is a failure in perspective.