One, it's almost a given these days that any secret program will eventually be rumored, revealed or detailed. So to rely on a program being secret (and how many contractors have access to all this?) is to design a security regime that is destined to fail.
Once a program is rumored, it's subject to possible question by Congress or others. In order to prolong the program's secrecy, security officials must lie. Yet the program will eventually out, revealing those lies, and further eroding our trust in government. If they lied about that, they're capable of lying about anything, by demonstration. The loss of trust is the most damaging result, since democracy depends on it.
Two, if a program is secret, generously funded, not subject to oversight (and if Congress can't talk about it, then there isn't any real oversight), that's a rich, fetid environment for the growth of sloppy procedures, illegal activity, and just plain waste. And remember, these are government contracts run by government contractors who lobby. The program will do anything it can to sustain itself.
Now that Prism et al. are out, the cover is blown. One of the justifications given for secrecy is so that targets are not aware of the surveillance, and are comfortable communicating in an insecure way. Are they going to disband the program and send everyone home? No. Notwithstanding any foiled terrorist plots (and how can we believe that, they're proven liars), the money has been largely wasted, and now the program that was designed to depend on secrecy will likely continue.
A better approach would be to assume the program will be revealed, and in fact reveal its existence openly in Congressional testimony. Keep secret what must be kept secret, like location of telecom vaults and other assets, but say what it is, what it does and what it's for.
This is much harder, because you have to design a program that will be effective even when known. But it's better, because its existence is acknowledged from the beginning, so citizens don't suffer from exposure eruptions and lies, and we know what's being done in our names to ourselves and everyone else. We even have some small chance of telling our representatives that we don't like it and to ask them to stop it. And if it's stopped, we have a small chance of believing that it really is stopped.
Any such program would likely be less sweeping than what's being revealed, and so less subject to abuse, and less potentially wasteful. People in the program will have to work harder and more creatively to get results, and they'll probably get better results.
If nothing else, the trend is not our friend, and if this kind of carte blanche isn't stopped then we eventually will look more like East Germany and less like the myths we learned in civics class.