You are thinking about this from the wrong direction. The problem isn't that the government hasn't provided a PKI, the problem is that social security numbers are being used for identification.
This was a serious fear when social security was originally created. For many years social security cards had the words "not for identification" printed on them. They contain no biometric data, not even a picture. Their purpose is not identification.
So the solution is obvious. Actually prohibit social security numbers from being used for identification. Don't allow creditors to even ask for them.
Then people will figure something else out on their own. Instead of a credit reporting agency existing at all, new credit applications could ask for your account numbers at existing creditors and then the new creditor can get your credit history directly from them. It would be straight forward to automate this -- and even require you to prove that you're the account holder by presenting your card from the other bank (or signing into its website if online).
There is no need for a national identification system. Having a bad one was the original problem. Replacing it with some differently bad one is no better.