Apple's git still trusts the compromised eMudhra certificates

Tangentially related: I fucking hate so much that browsers now all have their own DNS resolvers and caches, it's just Another Fucking Thing to debug when you're working especially on locally hosted stuff. This is not a browser's job, it's a task that should fall to the OS.

Certificate management is fifty kinds of all messed up - and in THEORY everything is designed perfectly and properly, in practice it's all bandaids on bandaids that nobody cares about as long as "things mostly work".

Is it possible for users to remove the eMudhra certificates themselves? I have no experience with MacOS, so I don't know if certain actions are prohibited. To be clear: I am not suggesting that normies do this! Rather, I am asking if it is possible for highly skilled users.

Better luck next time.

I have no idea how you've come to this conclusion, and clearly nothing of the sort is actually going on here.

I would've preferred a doc or a code repo link as a source. Otherwise, this just reads like a brag. A decent one, to be fair :)

    > Homebrew ships its own CA bundle

Outside of an enterprise environment, I never thought about having multiple "golden sources" of CA certs on a single host. (I have no experience working outside of classic enterprise environments, so please do not read my surprise as immediate dismissal of this idea!) For those unaware, in enterprise environments, it is very common to use a common CA cert pack from Oracle/Mozilla/Google/Microsoft/Amazon/Apple, but add 2-5 CA certs from your own company that act as trust roots.

I did some Googling on this matter. It looks like the "Big Four" of CA cert packs are: Mozilla, Microsoft, Apple, and Google. (I guess there is one or two in China also, be leave that alone for the moment.) What happens if they are in disagreement? This is a complex problem! If anyone is involved in this process, I would like to learn more.

So someone has to do it, and a pop-up on first boot asking you which "certificate providers" you want to trust would be as pointless as the "which browser do you want to use" ones were.

If I can speculate though on why these bundles come from the browser or OS and aren’t fully in the user’s responsibility. You’d have to start from scratch for it to not have a bunch of stuff the user had ‘no need for’ to be present, and the user would need to see a cert window semi-frequently that gives the possibility to start trusting a CA, and the bottom 99.99% of users in sophistication has no idea whether they should say yes or no. I mean, without being signed by a root CA you already trust, even the name on the new CA could be completely made up. “Do you want to trust the ‘Apple Root Certificate Authority’?”

There is. On macOS go to Keychain Access > System Roots and disable the kind of trust you want to disable. There's probably a scriptable method for that.

On Windows, open the management console to "Certificates". Then go through the list and move/remove/distrust any certificates you dislike. There are also powershell tools you can use.

On Linux, the situation is a bit annoying. You can remove all CA certificates manually and make the directory read-only to break any attempt from your distro to interfere with the list. You need to check your distro documentation to see if they use /etc/ca-certificates or /etc/ssl or something like /etc/pki for reference. Some distros offer a partial GUI as well.

Programs that deem themselves smarter than the OS (true in the case of macOS and various Linux distros), like browsers, the JVM or NodeJS applications, can have their own cert stores.

You will need to acknowledge half a dozen certificate prompts for every website you visit after this, though. Make sure to compare the fingerprints of the CA root certificate with the real ones you might find elsewhere, because it's easy to generate your own CA and copy all certificate information signed by your self-generated key.

- Only ISRG(Let's encrypt) being the sole CA and benevolent dictator of the internet. (then forced to be dissolved due to antitrust?)

- Each country controls their own CA and you need to purchase 200+ certs in order to make your website generally accessible. (then country XXX will charge you $200/year)

- Big techs owning OS/Browsers control their own CA and cross license each other (then Apple still being lazy handling incidents) (then they form a cartel and block others, like current Tier 1 ISP, email senders, credit card networks)

edit: put enter between numbered items.

The idea of the internet is that you can reach it across the world, globally.

I don't know why they haven't rolled out an update if they've already wrote a whole article about patching the problem yourself.

Dropping the Sectigo root along with the eMudhra root is the easy and safe solution, and maybe this is what Apple should have done (it’s the workaround they recommend!). But it means that some small percentage of certificates that are fine won’t be accepted.

Whatever reason Apple had for not updating /etc/ssl/cert.pem yet, I am certain it's not out of consideration for users that might have put custom certs in there.

I suspect the fast-track background security updates can only update the databases and revocation lists that live in the proprietary APIs, not any arbitrary file on the system

Then someone would ask "there has to be a better way" again, when the reason for the new inconvenience gets forgotten, and it oscillates.

For your tool, remember to nuke Windows' automated certificate authority updates (separate from normal Windows updates) and possibly the automated intermediary cache. Microsoft publishes a list of trusted CAs/intermediaries periodically and they get automatically applied to Windows so websites don't need to send the full cert chain during TLS negotiation, and it has a tendency to cache trusted intermediaries as well.

Mozilla's trust store is one of the first to get updated when a revocation happens, as well as Google's CRLsets. Downloading full CRLs repeatedly would be a waste of time and bandwidth, that's why there's a specific distribution method for CA updates. I would extend your tool with a daemon that checks for revocation automatically, so you don't accidentally eMudhra yourself while browsers and the OS get updates next time something like this happens.

This feels like all of the C code getting attacked now where everyone spent the last three decades building up cruft because fixing it would be a chore, and now the separate bills are coming due all at the same time.

Apple, for the company that prizes itself on believing privacy is a human right, why are you failing your customers so badly on PKI management?

I wonder why they stopped doing that and now just seem to do "standard" builds and end up with this (at least) 2 separate trust store situation.

There are people from these CAs who were denied in the past explicitly for concerns about spying who now work at eMudhra[3,4].

    0: https://bugzilla.mozilla.org/show_bug.cgi?id=1474556
    1: https://bugzilla.mozilla.org/show_bug.cgi?id=1427262
    2: https://groups.google.com/g/mozilla.dev.security.policy/c/nnLVNfqgz7g/m/TseYqDzaDAAJ
    3: https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c43
    4: https://emudhra.com/en/blog/scott-rea-joins-emudhra-as-senior-vice-president-global-strategic-advisor

https://support.apple.com/en-us/102657

Similar reason (though with the opposite effect) as how a cert can be fine for browsers but fail for curl when they don't send an intermediate: the "vanilla"/library/command-line stuff is just checking for a path to a trusted cert and not doing the CRL or AIA stuff a browser does.

yeah, it's trivial to punch somebody in the face in the morning, all you need to do is to be by their bad in the morning when they are waking up.

If you get past all of that, you still need a victim who is actively using a cli tool on macOS using https.

It’s not impossible, but the conditions are much more specific than those needed to inject porn into a webpage on a poorly configured school network in 2006.

Yeah that's not so much "trivially easy"

So this isn't like Firesheep [0]. Yes, there would be vulnerable traffic, like the git client this thread is discusing, but it's not "everything"

[0] https://en.wikipedia.org/wiki/Firesheep

I’d also bet more things use libcurl behind the scenes than we appreciate, and that at least a few of them have things like silent updates which retry after a failure without notifying a user. I’ve seen a lot of agents which just try to periodically check in because they’re designed not to be noisy on lousy networks.

It's not like they cannot introduce a bad certificate right now. We would be just be taking the ability from all other countries from doing the same.

I don't think its wise to make current security decisions based on screw ups from almost 2 decades ago.

If that's the worst you have against Linux then it looks like it has done very well.

Upstream was involved in it as well. Sure it was a communication problem, but it's not like the maintainer just decided to this unilaterally. Also this WAS undefined behaviour, so from a language perspective it was already broken, even if it worked accidentally in practice.

Morality doesn't solely depend on action or inaction, it also matter what you act or don't act to.

Besides, how does this relate to my Linux point, where no one installs anything without you knowing it?

And there's a lack of motivated attackers for a reason, the attack scales poorly. Maybe you can bamboozle a target or two, but if you go for hundreds, somebody's gonna start wondering what's up with the keys. If you're trying to take over the world, or sell to somebody who is, you're better off focusing on compromising a root of trust or tampering with name resolution, because attacks like that are sustainable at scale.

That is unequivocally bad, similar to Java which ships its own keystore.jks and many applications shipping its own copy on top of that. True nightmare fodder.

Yes, and that makes not dealing with this issue worse on Apple's part. If MacOS' setup somehow doesn't allow untangling a (still deemed trustworthy?!?) root certificate issued by eMudhra from the broken intermediate certificates, then revoke that whole chain including the root & accept collateral damage.

Failing that just provides an incentive for apps to do their own chain-of-trust implementations. Which like you said is not the way to go.

In reality it's macOS shipping the outdated certificates, and it affects various other command-line tools, like Curl.

"Ten days later, Apple Image Viewer on macOS still trusts the compromised eMudhra certificates" doesn't sound as catchy, but is equally correct because the apps use system CA.