In between those extremes you present, there are the huge number of people who at one point or other does something that might make them blackmail targets if some future administration needs something from them, their family, their friends or their employer, or that have carried out any kind of activism or political or religious activity that might brand them in the future.
We have two recent demonstrations of why people should be concerned about this: McCarthyism and 9/11. Imagine McCarthy with access to years worth of stored electronic communications. But you got rid of that. Instead you got post-9/11 restrictions in the dark like the no-fly list. Imagine the post-9/11 attitudes to muslims escalating one step further. It takes very little before a lot of seemingly uninteresting stuff that nobody cares much about now is suddenly enough to ruin careers or relationships.
It takes very little before the impact of indiscretions online or on the phone can be used as a source to impose even more restrictions on you. Suddenly some people will get fewer job offers for example.
There are much worse possibilities. But these are just simple, straight forward escalations similar to episodes in the recent past.
For my own part, about 20 years ago I was politically active in a left wing political party where just a few years before mere membership was often sufficient that you'd get denied entry to the US. Despite the fact that membership lists were confidential and never released to outside sources.
But then again all officers of the party had known they'd been under extensive surveillance for many years (yet were considered crackpots for claiming so, until it was proven the Norwegian security services had engaged in extensive illegal surveillance, including against a left wing member of parliament on the parliamentary commission investigating their violations while they were investigating them)
The groups that western intelligence organization target have shifted, but I'd be outright shocked if there are not perfectly legal, democratic organisations where membership would cause restrictions on your air travel and trigger additional surveillance without any evidence of even potential threats (some members of the party I was involved with at various points where shadowed on the way to and from work routinely, and "ran into" surveillance people that taunted them about private conversations they'd had in their own home...)
This is the flaw with the idea that you don't have anything to hide. Everyone has lots to hide - as much as it doesn't look that way, we all filter and none of us post everything in our lives on Facebook. We just don't think we have anything to hide because many haven't done anything they consider to be wrong or believes the current government will care about. Guess what? Most of the people targeted by Joseph McCarthy didn't think they had anything to hide either, and most didn't until his witch hunts started, yet many of them lost their jobs and got blacklisted.
This is the really scary thing about these NSA revelations: The surveillance is bad. But the storage of data over time is 10 times worse - it means the threat is not just abuse now, but abuse from the next government too, and we don't yet know whether that government will be better or worse in terms of how it might use or abuse that data, or if they might simply decide to extend storage periods further and put people at further risk of even later governments.