You visit the bank in person and the bank gives you the keys in a flash drive, QR code, printed paper, ...
Then you go home and install the keys
The role CAs play is to be a trusted identity verifier so we don't have to have millions of people driving around to do key exchanges in person.
I think the idea of a CA is good but it should be distributed somehow
CAs are quite distributed already. You probably have a couple dozen or even a hundred different root CAs installed right now, and you can install whatever ones you like if necessary.