well not using PGP specifically, but imagine having a social security card with two QR codes on it in addition to your social security number. one of the QR codes contains a private key and the other a public key. The financial institutions and credit reporting agencies can freely access your public key and it's safe to give away. You can make signatures with your private key when it's scanned at a bank or on a phone and the signatures can be verified to be correct by your publicly available public key.

I like the idea better of making additional keypairs that have a chain of signatures back to your social security card so that you don't have to rely on it as much. It seems to me there's a lot of things that could be very workable as far as this is concern, but just to be clear I just like to use PGP as an analogy to a system that could work.