They were doing that exact thing to selected citizens. So did FBI for alleged communists.
Bugging apartments, planting their own spies, opening up private letters, etc.
Yup, they wouldn't even dream about that magnitude.
It's the difference between being able to point a gun at a few inconvenient people, and being able to point it at everybody, all the time. Eventually the temptation presented by this kind of power and control over people will intersect with some Erdogan or Lukashenko type with the finger on the trigger.
But that makes it worse, not better - those who believe in their ideology with such zeal are more likely to push it through to the extremes, no matter what.
Why is it always "them" vs "us"? NSA/CIA are doing probably a better job at global surveillance of everything and everyone, simply based on resources available to them. Every powerful enough government spies on the largest scale they can, because just letting someone else to spy on you and not spy yourself (if you can) would be infinitely foolish.
Even at local scale. If citizens could find a way to hold their president on short leash, such president would be more likely to listen to the citizens. But citizens don't have such powers, only presidents do, so they only have to listen to each other.
Similarly for KGB; I believe that standard practice was to recruit secret informants e.g. from every course group in universities, so any student activities would be fully covered and you could get a personal report about every student; for 100% of any trips abroad the group had an agent/informant or multiple within the group. So you'd have a small fraction (multiple percent?) of population actively involved in the monitoring, and a majority of people being at least occasionally monitored - as the archives later revealed, an average person should expect that your colleagues, friends and relatives will [have to] write reports on you (corroborated by the fact that any group might include two or more informants, so any omissions would be obvious and cause consequences for the false reporter) and note any expressions of anti-government sentiment.
The aim was to effectively keep a gun aimed at everyone, and they were somewhat successful in achieving that (or at least maintaining that belief) even without modern technologies.
The aim was to make everyone believe there may be a gun pointed at them even if there wasn't. While that was relatively effective do you know what is even more effective? Actually having a gun pointed at everyone.
Having 500k Stasi informants and spies over its 50 years of existence is a far cry from being able to effectively spy on everyone all the time. And informants are notoriously unreliable due to a whole host of human failings, not least among them being that the people want to gain favor or at least not antagonize the state.
The modern surveillance state would make you inform on yourself via every device or communication channel you use, and at all times, with some human/AI combination making sure very little if anything is missed.
But they couldn't do it to everyone. Now with computers they can. You might think 1984 was written as a warning, but the powers that be see it as an instruction manual.