There's some decentralization within the government also, if one branch wants something from another it (ideally) has to go the courts route too, it's not that every federal employee can just access any government records on a person, (at least in theory). If there's someone powerful enough taking interest in you to override that, I doubt a private bank is going to be a major obstacle.
I mean courts themselves are technically part of that system too. I am as cynical as the next person regarding government overreach, but I see little protection from a multi-national banking entity entangled with the government for decades in all sorts of ways.
If it was as 'easy' to start a bank as an ISP in the 90s, I could see it.