Hacker News Reader

> CryptoAPI, for its warts, is actually beautifully engineered, in that Microsoft has had the ability to add arbitrary constraints and properties to certificates from the very first release (via CERT_PROP_IDs). You don’t really see these in the UI; you only find out about them in WinCrypt.h, or via debugger stepping. However, it means that even if the UI is showing “trusted for TLS”, Microsoft may have disabled trust for TLS via the extended properties, which their APIs respect, and which are delivered through authroots.cab

I mean no disrespect, but if you need to look through header files or use step-through debugging to actually find out what constraints apply to a certificate, especially when the ones actually reported for the certificate don't actually apply in reality then "beautifully engineered" doesn't seem like the right adjective to use.

edit: I don't see this as a documentation issue; when the only interface you provide the user says one thing but stepping through the code reveals that that's entirely not true for certain items in certain circumstances, then having a better ReadMe is not the issue here.

I suppose it’s a question of whether you count “documentation” as engineering.

It is elegantly complex and featureful. And (sometimes intentionally) poorly documented, as much of that implementation is seen as “an implementation detail”.

Especially when it comes to API's, documentation is 100% part of the engineering. Very few non-trivial API's are self-explanatory because you have to map a complex problem space to an imperfect engineering-focused formulation of the problem domain, and the mapping usually involves introducing design patterns to make the problem space easier to manage in code.

It can be beautifully engineered yet poorly documented. It happens to quite a lot of software actually.

I think that more documentation needs to give context about design decisions/principles and why they were made. And hopefully the software is sticking to some principles. If you can learn a small number of orthogonal principles, you can predict the behavior of the software without consulting the documentation, ideally

As someone who's actually had to use the CryptoApi, I think it's unpleasant to use, and often the documentation isn't good enough. There is also usually at least 4 different ways of getting to the same result, which makes things very confusing. And some features are only supported from Windows 7/10.

"Beautifully engineered" is not the phrase that comes to mind.

CryptoApi is remarkably flexible and also really hard to use for a bunch of reasons:

1. It does a lot of things. It's an OS API for talking to cryptographic hardware, including the TPM and smart cards, in addition to the API for doing run of the mill cryptographic operations using the OS's implementation. That's on top of serving as a primary way to load/use certificates (through the Ncrypt API, but often you can't entirely avoid dipping into CryptoAPI).

2. It's really, really low level, meaning that you have to directly deal with the intersection of driver/hardware behaviors, feature matrices, and data structure serialization/packing.

3. It's actually two API's merged into the same set of function calls: CAPI, and CNG. Depending on which API is supported for your usage or supported by the hardware drivers, the behavior of the same API call might be subtly different.

4. It's a C API with the classic Win32 API structure packing conventions, including the really fun conventions like "allocate the buffer, pack the data structure, and set the offsets yourself without an API to do it for you" convention, and also the "call a function with a null data pointer to get the size of the buffer you need, and then allocate that buffer and call it again to actually do the work" convention.

5. Because it's so low level and nobody uses it, there's very little documentation on how to troubleshoot errors when you do things wrong, and the API is complex enough that you will do it wrong, and then you will spend a few days just trial-and-error-ing your way out of that hole. There are a lot of places in the API's where in order to know how to do something, you need to look up the behavior of the component you're talking to on the device/provider side of the API, but those providers are often under-documented.

Most developers should try to avoid using it because it's so easy to mess up. The API is extremely powerful, but it's also more of a footgun than it should be.

I know it's not usually a thing to reply to your own comment, but I wanted to add an important detail.

The reason for point #1 is that doing all of these things in one API is a useful framing of the problem. For example, if you want to just use a certificate or cryptographic key pair regardless of whether the private key is in a file or stored in a hardware module, CryptoAPI unifies that interface so that, if you do it right, you have two orthogonal pieces of code: one that locates the key to use and which adapter to use to talk to the underlying software/hardware, and another to request cryptographic operations on an abstracted key pair. The fact that you're talking to pure software or an actual HSM can be abstracted from the code that needs to sign a challenge.

The API is powerful enough that its abstractions work even for cloud based key storage. For example, AWS Cloud HSM support for CryptoAPI: https://docs.aws.amazon.com/cloudhsm/latest/userguide/ksp-li...

It could certainly be the case that it's poorly designed, but it's not necessarily the case just because they've hidden the feature from the UX.

First, the backend constraints may be ideal despite whether they've bothered exposing any of them in the UI. Second, not exposing them in the UI may have been an explicit design decision and not just trade-offs. It's possible that a root certificate list was deemed complicated enough without adding a zillion more hard-to-debug toggles that might allow curious users to accidentally break something in a way that would be very confusing, and only allowing programmatic access was deemed to be best because likely the settings would only be used by enterprises managing their fleet or some such.

Even if the policy data is absent from UI for legitimate reasons, if the policy can't be programmatically extracted, the documentation can't be trusted.

That is, the policy should be a declarative object interpreted by a well-tested engine, or a small simple bit of logic, not a crawl through the engine code looking for 'if' statements.

Eh, my point was just that:

  - Policy is largely encapsulated on the certificate properties in the root store
    - Local Policy is implemented via registry keys, with somewhere like 100+ odd registry keys (... many undocumented, as they implement customer-specific features whose documentation is provided under a MSFT support agreement)
  - Policy is a mix of documented (via WinCrypt.h) and undocumented flags
    - The reason for not documenting some of this is that they’re flags that Microsoft may or may not support; that is, they’re implementation details
    - CAPI provides, by design, several ways to fully replace their policies, either for single trust purposes or for all
  - The UI exposes a suitable “general purpose” expression

I can totally understand the complaint that this isn’t clear in the UI, and as others have noted, sometimes that’s intentional.

I can also totally understand the complaint that this isn’t all meticulously documented. Not everyone can ring up their pet Microsoft engineer to get documentation and show how it connects to a problem that Microsoft benefits from helping us solve (the linked-to crt.sh bug, which as a side-effect helps provide greater automation for Microsoft and the CAs they supervise). However, Microsoft also, from the get-go, designed it to be extensible so you could replace this.

Importantly, Microsoft has been able to roll out new features, and remove trust in CAs, without major re-engineering work. That was the “beautiful engineering” part of my comment. They defined a stable API that could be easily extended, or even wholesale replaced, in the Windows 2000/XP era. That API has brought considerable improvements WITHOUT requiring rewriting your code for Windows Vista, 7, 8, or 10 to leverage that. That’s a considerable difference from some other libraries and tools; LibreSSL was hugely constrained fixing OpenSSLs broken chain building, Mozilla Firefox/NSS has undergone three complete and separate rewrites from the ground up of the engine, Apple deprecated dozens of APIs when they ported iOS’s verifier to macOS, etc. As an engineer, 20+ years of stability IS impressive!

Having recently implemented some new product features using several parts of the CryptoAPI, my experience agrees. There were several instances of taking days of dead-ends to get something to work the way you wanted. Eventually finding the correct way through trial and error or some obscure example code on the internet from 10 years ago. But once you cracked it, the code and API felt like it would be rock-solid for decades to come.

It’s a mess of discoverability and documentation, but as someone else in this thread said, beautifully engineered.

> Mozilla Firefox/NSS has undergone three complete and separate rewrites from the ground up of the engine

Can you provide more details? I'm only aware of one of these rewrites...

1. Legacy

2. Trust Domains (half completed, but littered throughout the code as nss3 prefix). This was being lead by Sun and stopped when Oracle acquired them.

3. libpkix: This was done by porting Java code to C using preprocessor macros to simulate exception handling. It implemented path discovery, and not just verification, and was used by Firefox for EV processing (AIUI), and was always used by Chrome on Linux/ChromeOS (until recently replaced with the Chromium built-in verifier)

4. mozilla::pkix, which started off as Brian Smith’s insanity::pkix rewrite of a minimalist path builder/verifier. I can’t remember if this launched while Brian was still at Mozilla or after he had left, but Brian would later take the approach he used for insanity::pkix when writing the Rust webpki project ( https://github.com/briansmith/webpki )

Each of the above APIs had significantly different interfaces for controlling verification. The trust domain stuff wasn’t as visible, because it was only half-completed.

Ah good, so mozilla::pkix is the newest one and I don't have to worry about that being replaced by something else instead.

The approach Microsoft took here encourages something we both I think agree is a bad idea, roots used for multiple potentially conflicting purposes. The owner of the multi-purpose root will either make a big fuss when policy changes down the road make that conflict painful or worse they'll persuade themselves that we can't really have meant what we just told them because it's so painful, "obviously" the policy change doesn't apply to their multi-purpose root and they'll ignore it until discovered.

If for example Microsoft had separate "Code signing" and "Web PKI" trust programmes then you get two good things for the price of one: Less incentive for CAs to do things we'll regret even if they don't; and clarity for users about the purpose of the trust relationship.

Mozilla could do better here too, it's actually not easy to discern from Firefox which built-in CA roots are actually trusted in the Web PKI, and it's outright impossible as far as I know to discover any of the special case restrictions described in https://wiki.mozilla.org/CA/Additional_Trust_Changes from the user interface of the software itself.

> roots used for multiple potentially conflicting purposes

Wouldn't this lead to a proliferation of protocol/purpose-specific roots, to the extent that there might be so many critical roots that administrators become careless about vetting requests to add a root, or audit logs that a root is added?

It feels like the solution is a better UI to visualize the many-to-many relationship between roots and protocols/purposes, rather than requiring every root to be single-purpose.

I think you’re right for questioning the end-state, but you may be missing the current status quo.

The current status quo of multi-purpose roots is that supervision of a root, by a browser, auditor, and the CA themselves, constantly has to consider all the purposes a certificate is trusted for when considering the effective policies and design.

The classic example I give is that multiple CAs would have policies like “If it’s intended for TLS server auth, we put the domain name in the CN field and make sure it’s a real domain name. If it’s for document signing, we put the company name in the CN field and make sure it’s a real company name”, and with no further distinction between the two. So what happens when you have a company named “pets.com” - did they validate the domain or not?

What about if they misissued a TLS certificate. For TLS, you can quantify the impact, and you can move to distrust the CA, only thinking about how TLS works in your application. But what if they also issued certificates for lawyers used in judicial proceedings, and rely on the CA being trusted in the OS? This was the exact problem with DigiNotar, and why it took Microsoft over a month to fully respond, where Google and Mozilla took days.

We see similar issues with audits, and CAs having sub-CAs not “intended” to issue a particular type of certificate, but totally doing so.

For lack of a better analogy, and since earlier in this post I was talking about API design, the choice of singular trust purposes is like the S in SOLID. You want a single responsibility, clear, and do that well. It makes it easier to change and evolve that API when you don’t have to worry about breaking 30 odd unrelated use cases. And it lets you narrowly focus the supervision to the task at hand.

Pretty bad that “trusted” could mean blocked and it was only possible to know this in Windows 10

So it’s hidden magic, yet the EULA states you are responsible for choosing which roots to trust.

If you can't even remove CAs from the trust store without breaking it, then it doesn't fucking work, does it?