HNHacker News
TopNewBestAskShowJobs

jaas

5,640 karma · joined May 12, 2016

Executive Director @ ISRG. Let's Encrypt, Divvi Up, Prossimo.
submissionscomments
jaas··on Rustls Server-Side Performance
The repository is here:

https://github.com/rustls/rustls-openssl-compat

We just work with normal issues/PRs, and there is a Rustls discord channel if you want to chat. We'd love your help!

jaas··on Rustls Server-Side Performance
Rustls uses aws-lc-rs for cryptography, which, roughly speaking, is based on the cryptography from BoringSSL, which is a heavily modified fork of OpenSSL from a long time ago. I'm not sure how similar OpenSSL and aws-lc-rs cryptography implementations are today (maybe someone else knows?), but it's probably not accurate in a useful way to say that aws-lc-rs just uses cryptography from OpenSSL.

In any case, OpenSSL does a whole bunch of things, and one of those is providing low-level cryptographic routines. When people talk about issues with OpenSSL, they're usually not (in my experience) talking about issues with its low-level cryptographic routines. They're talking about things like the TLS implementation and API.

Rustls has its own Rust code for the TLS protocol and certificate parsing/validation, which doesn't come, directly or by lineage, from OpenSSL or any OpenSSL derivatives.

jaas··on Rustls Server-Side Performance
I think what you're quoting says what I was saying - assembly with some C around it, wrapped in a Rust API. At least for the "select" (read: most important) algorithms. The details of the C/asm boundary in aws-lc are hard to summarize.

Ideally the C would eventually move to Rust, but I think aws-lc needs to work in many contexts where a Rust toolchain is not available so it might be a while.

Graviola is an interesting option under development, in part because it gets rid of the C:

https://github.com/ctz/graviola

jaas··on Rustls Server-Side Performance
In Rustls, DER, and all certificate parsing and validation in general, is done in Rust.

https://github.com/rustls/webpki

jaas··on Rustls Server-Side Performance
In Rustls, TLS is implemented entirely in Rust. It uses aws-lc-rs [1] for cryptography, and aws-lc-rs uses assembly for core cryptographic routines, which are wrapped in some C code, which then exposes a Rust API which Rustls uses.

It's not practical right now to write high performance cryptographic code in a secure way (e.g. without side channels) in anything other than assembly.

[1] https://github.com/aws/aws-lc-rs

jaas··on Rustls Server-Side Performance
This report contains more details about the results discussed in the blog post:

https://rustls.dev/perf/2024-11-28-threading/

jaas··on Rustls Server-Side Performance
I'm the person who is running the rav1d bounty, also involved with the Rustls project.

In many (most?) situations I think Rust is effectively as fast as C, but it's not a given. They're close enough that depending on the situation, one can be faster than the other.

If you told me I had to make a larger and more complex piece of code fast though, I'd pick Rust. Because of the rules that the Rust compiler enforces, it's easier to have confidence in the correctness of your code and that really frees you up when you're dealing with multi-threaded or algorithmic complexity. For example, you can be more confident about how little locking you can get away with, what the minimum amount of time is that you need to keep some memory around, or what exact state is possible and thus needs to be handled at any given time.

There are some things that make rav1d performance particularly challenging. For example - unlike Rustls, which was written from the start in Rust, rav1d is largely the result of C to Rust translation. This means the code is mostly not idiomatic Rust, and the Rust compiler is generally more optimized for idiomatic Rust. How much this particular issue contributes to the performance gap I don't know, but it's a suspect and to the extent that it's worth pursuing, one would probably want to figure out where being idiomatic matters most instead of blanket rewriting everything.

jaas··on Rustls Server-Side Performance
Rustls has two C APIs.

The first is C bindings for the native Rustls API. This should work great for anyone who wants to use Rustls from C, but it means writing to the Rustls API.

The second is C bindings that provide OpenSSL compatibility. This only supports a subset of the OpenSSL API (enough for Nginx but not yet HAProxy, for example), so not everything that uses OpenSSL will work with the Rustls OpenSSL compatibility layer yet. We are actively improving the amount of OpenSSL API surface that we support.

jaas··on Rustls Server-Side Performance
The default cryptographic back-end for Rustls, aws-lc-rs, is FIPS compliant and integrated in a FIPS-compliant way so it's easy to get FIPS compliance with Rustls.
jaas··on $20K Bounty Offered for Optimizing Rust Code in Rav1d AV1 Decoder
This is the correct answer (we run this bounty). Contests can be legally complex, there are only so many place we feel comfortable running it from a legal POV.
jaas··on The State of SSL Stacks
> We also tested Rustls and its rustls-openssl-compat layer. Rustls could be an interesting library in the future, but the OpenSSL compatibility application binary interface (ABI) was not complete enough to make it work correctly with HAProxy in its current state.

Rustls is actively working on improving the OpenSSL compatibility layer. Hopefully we'll have it fully working for HAProxy soon!

We've also invested a lot in performance. Next week we'll be publishing a blog post about Rustls server side performance. Since it's relevant to discussions about TLS stack performance, here's a preview:

https://docs.google.com/document/d/1xFoRjb7pn4ZtL5BH7_ZwXNgN...

jaas··on Rust’s dependencies are starting to worry me
I don't think an additional standard library layer, whatever you call it, has to have the same tight controls on backwards compatibility and evolution that the actual standard library has. IMO the goal of creating it should be to improve supply chain security, not to provide an extremely stable API, which might be more of a priority at lower levels but chokes off the kind of evolution that will be needed.

I think what you're suggesting is a great idea for a new standard library layer, you're just not using that label. A set of packages in a Rust namespace, maintained by the same community of folks but under policies that comply with best practices for security and some additional support to meet those best practices. The crates shouldn't be required, so no_std should work just as it would prior to such a collection.

jaas··on Let's Encrypt is 10 years old now
We consider our ten year anniversary to be in 2025 but I appreciate the kind words here!

Today is roughly the ten year anniversary of when we publicly announced our intention to launch Let's Encrypt, but next year is the ten year anniversary of when Let's Encrypt actually issued its first certificate:

https://letsencrypt.org/2015/09/14/our-first-cert/

In December of 2015 (~9 years ago today) is was made available to everyone, no invitation needed:

https://letsencrypt.org/2015/12/03/entering-public-beta/

jaas··on Jeremy Rowley resigns from DigiCert due to mass-revocation incident
No team is going to prevent issues like this 100% of the time. That's a wildly unrealistic expectation.

Wherever the bar is, it won't be 100%. That's why good leadership invests in the ability to respond well to mistakes that will inevitably be made.

jaas··on Intent to end OCSP service
Yes, I disagree. Best case scenario I think it would just allow us to get rid of the CDN layer. We'd still have to build and manage the rest with utmost care.

Even that really depends on what the "SLA" expectation is. How many QPS would we be required to support before we're allowed to start dropping queries? If there's any possibility of a burst beyond what we can handle locally we'd keep the CDN, so maybe the CDN bill would be significantly smaller on account of lower volume but all the complexity and care of operation would still be there.

jaas··on Intent to end OCSP service
OCSP systems at scale are complex. At the core there is an on-demand or pre-signed OCSP response generation system, then there is usually an internal caching layer (redis or similar), then there is an external CDN layer. Just because the outer layer appears not to know about a certificate (a bug, to be sure), doesn't necessarily mean that other CA systems don't know about it.

Requiring CAs to run OCSP makes running a CA more complex and expensive, which has considerable downsides like it does for any system, particularly as it is a zero sum game. Every dollar or hour of engineering time that Let's Encrypt spends on OCSP (and it is a considerable amount) is a dollar or hour not being spent on other parts of the CA. From the outside that may not be very visible because it's not obvious what isn't getting done instead but there is a real and considerable cost that is not worth it IMO.

jaas··on More Memory Safety for Let's Encrypt: Deploying ntpd-rs
People bring up postfix all the time in this context because supposedly nobody has ever found a memory safety vulnerability in it. Presumably this is supposed to make the point that it is possible to write complex programs in C safely.

The reason people know this about postfix and keep bringing this one specific example up is because it's so unusual! It's an example that almost stands alone, it's extraordinary.

I don't think this is making the point about the safety of C that you think it is.

There is a mountain of evidence suggesting that C is dangerous, particularly for network services, and one possible example to the contrary doesn't change that.

jaas··on More Memory Safety for Let's Encrypt: Deploying ntpd-rs
ntpd-rs support NTS, I agree it would be great if more people used it!
jaas··on More Memory Safety for Let's Encrypt: Deploying ntpd-rs
I'm the person driving this.

NTP is worth moving to a memory safe language but of course it's not the single most critical thing in our entire stack to make memory safe. I don't think anyone is claiming that. It's simply the first component that got to production status, a good place to start.

NTP is a component worth moving to a memory safe language because it's a widely used critical service on a network boundary. A quick Google for NTP vulnerabilities will show you that there are plenty of memory safety vulnerabilities lurking in C NTP implementations:

https://www.cvedetails.com/vulnerability-list/vendor_id-2153...

Some of these are severe, some aren't. It's only a matter of time though until another severe one pops up.

I don't think any critical service on a network boundary should be written in C/C++, we know too much at this point to think that's a good idea. It will take a while to change that across the board though.

If I had to pick the most important thing in the context of Let's Encrypt to move to a memory safe language it would be DNS. We have been investing heavily in Hickory DNS but it's not ready for production at Let's Encrypt yet (our usage of DNS is a bit more complex than the average use case).

https://github.com/hickory-dns/hickory-dns

Work is proceeding at a rapid pace and I expect Hickory DNS to be deployed at Let's Encrypt in 2025.

jaas··on Memory Safe TLS Library Now Has AWS Crypto and FIPS
I assume you're asking why the underlying crypto still needs to be written in asm. There are two primary reasons:

1. Performance

2. Defense against side channel attacks (e.g. constant time operations)

jaas··on Memory Safe TLS Library Now Has AWS Crypto and FIPS
The Rustls TLS implementation and certificate verification are all safe Rust.

The underlying cryptography is still a mix of C and asm, that's the best option we have now particularly if we want support for things that make it deployable, like FIPS. We are looking for ways to improve the safety of the underlying crypto in the future.

jaas··on You Paid $1k for an iPhone, but Apple Still Controls It
Because thieves know that all iPhones are locked like this and thus don’t steal them.

If they knew that some significant percentage were not locked they would be worth stealing again, and if it’s locked they just throw it in a dumpster.

The fact that your phone is locked protects everyone else as well.

jaas··on The database servers powering Let's Encrypt (2021)
This did not cost us $225k. About half that. Nobody pays the website price, you pay a lot less via a VAR.

- ED of ISRG / Let's Encrypt

jaas··on Shortening the Let's Encrypt chain of trust
Those numbers are misleading. IdenTrust has always been a small CA in terms of volume. The large percentage you see for them there is actually Let's Encrypt volume counted as IdenTrust because of the cross-sign. The Let's Encrypt percentage there is from sites not using the cross-sign. Add them together and that is basically Let's Encrypt's total volume, as IdenTrust itself is likely < 1%.
jaas··on Shortening the Let's Encrypt chain of trust
As the person who negotiated the agreements between Let's Encrypt and Identrust I can tell you that they have provided valuable services, including but not limited to cross-signs. I would not describe it as rent seeking.

We are sincerely glad to have them as partners, and grateful for their contributions to helping get Let's Encrypt going. We could not have done what we did without them. Running a publicly trusted CA is not easy, and cross-signing others involves work and liability, particularly if the entity asking for a cross-sign is an upstart with a strange plan and little to no experience running a CA.

jaas··on Ask HN: Who is hiring? (July 2023)
ISRG / Let's Encrypt | General Counsel | Half Time | Fully Remote

If you’re an attorney looking for a flexible half-time position with great work/life balance and a mission that matters, this could be the job for you!

Internet Security Research Group (ISRG) has changed the Internet for nearly everyone using it. You’d be our next staff member (our 24th), joining a highly motivated and future-focused organization. We make the Internet more secure and privacy-respecting. Our largest project, Let’s Encrypt, helps more than 300 million websites protect their visitors. We operate as a 501(c)(3) nonprofit so we can stay focused on what’s best for the people using the Internet. This model lets us take on tough problems and we’ve built up the staff capability to do it.

https://boards.greenhouse.io/isrg/jobs/4247229006

jaas··on Calling time on DNSSEC: The costs exceed the benefits
You can read more about multi-perspective validation here:

https://www.cs.princeton.edu/~jrex/papers/multiva20.pdf

jaas··on Calling time on DNSSEC: The costs exceed the benefits
Thanks in large part to our research partnership with the group at Princeton CITP, Let's Encrypt has deployed a system we call multi-perspective validation. We validate domain control from multiple network perspectives so that an attacker would have to pull off several simultaneously successful BGP attacks in order to hijack our validation. It's not perfect but it makes it very difficult to do. There are other improvements coming that will further improve effectiveness.

https://www.cs.princeton.edu/~jrex/papers/multiva20.pdf

I believe the Google Trust Services CA recently announced that they have started doing this as well.

jaas··on Rustls TLS Library 0.21.0 Released with New Features
There is a C API for anyone who wants to use Rustls from C:

https://github.com/rustls/rustls-ffi/

jaas··on Rustls TLS Library 0.21.0 Released with New Features
I edited the title to note that it is a TLS library. Apologies for any confusion!
← PreviousPage 2 of 9Next →