I don't think it's necessarily a dealbreaker if you consider this: from a purely technical standpoint, there's nothing really stopping anyone from setting up a certificate authority- the only issue is getting service providers to trust it enough to accept those client certs as sufficient identification. I could easily imagine a world where I receive an "official" client cert from a government (which I can use to thoroughly prove my identity if needed) as well as several "pseudonymous" certs from various other CAs that I may use from time to time.
The main difference between CAs would be the kind of attestations they provide for a given certificate holder. For example, I could imagine a CA which (for example) is set up to attest that any holder of a certificate signed by them is a medical doctor, but will not (by policy) divulge any additional information.
Or perhaps a CA which acts as a judge of good character- they may issue pseudonymous or anonymous certs, but provide a way for application owners to complain about the behavior of a user presenting that cert.
I'm sure there are plenty of holes that can be poked in this model but I don't think it'd be completely out of the question?