Opponents mention that counter-terrorism is not effective for all cases of terrorism, in effect arguing for an increased and more efficient surveillance. "They were already on some watch-list.", of course, mass surveillance put them there successfully.
I hear of surveillance abuse, where privacy is violated (spying on ex-girlfriend). But, like the police has a monopoly on violence and can physically restrain your freedom, so has the government a monopoly on violence of privacy. If you deem your country incapable of holding that responsibility, I guess it is time to move to a country where the US government can gather even more than your telephone meta-data.
Finally, that we could end up in a totalitarian surveillance state, is a pessimistic projection akin to adversarial AGI or all police deciding to start abusing and shooting random citizens.