This really is a key quote. Even if OP's assertions about the NSA are totally correct, even if all security protocols are followed to the letter, the problem still remains that they have a tremendous amount of power that can be used to target anyone deemed an enemy of the state.
I think a lot of contention on this issue revolves around how much you trust the government to appropriately designate enemies of the state. Many people believe the government is responsible about this, and that it will only go after people who a reasonable person would consider "dangerous." The problem, of course, is that the United States doesn't exactly have the cleanest track record of appropriately focusing its wrath:
http://en.wikipedia.org/wiki/COINTELPRO
(And yes, I know COINTELPRO was FBI, not NSA... I believe it's still an instructive example of government overreach.)
Anyone who defends the NSA on the grounds that it only targets those who are worthy of targeting needs to convince me that another COINTELPRO will never happen. I would actually welcome such an argument, since it would make me feel a whole lot better about this.