HNHacker News
TopNewBestAskShowJobs

julian-klode

171 karma · joined December 28, 2015

submissionscomments
julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
Also, sure, the key type is also stored with the public key, and then the signature type must find a public key for the same type; so it looks like ed25519:<base64>.
julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
I think there's a reasonable compromise of embedding the key id of the primary key, but shipping the subkey completely.

This way we lose the ability for checking total consistency without public keys present; but we have the technological requirement you want for the user to trust the primary key (and no way to construct it from the signature).

We discussed the signature format in great detail, and looked at multiple options:

  Ed25519-Signatures:
   base64<sig1>
   base64<sig2>

  Signatures:
    ed25519:base64<sig>

  Signatures:
    ed25519 base64<sig>

  Signatures:
    base64<ed25519 | sig>
The first option was rejected by people for reasons I can't remember, and the next 2 are equivalent, albeit the one with : is easier to copy-paste (who does that for an apt signature?).

The last one we did not like because it made the code more complex because we'd have to decode to a buffer first and then copy into the appropriate struct; and also because it's not clear from looking at the plaintext which algorithms it has been signed with.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
Not gonna have two different signing implementation. Same security concerns apply to Debian too, of course.

Having 130+ packages to rebuild for a critical security update is no fun.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
You have the choice between GnuPG 1 and GnuPG 2 :D
julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
Rust is a security nightmare. We'd need to add over 130 packages to main for sequioa, and then we'd need to rebuild them all each time one of them needs a security update.
julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
The point is not about valid signatures from untrusted keys, it's about invalid signatures from untrusted keys.

Some repositories are signed with multiple keys. People building the repositories might only have one key T as trusted, and then mess up signing with the other key U (imagine that one is an offline key - this is how debian stable releases are signed).

By verifying all signatures and rejecting any invalid one, it will be immediately obvious if the signature by U is wrong.

Also, as mentioned before, the scheme allows for rapid subkey replacement in case of potentially compromised subkey.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
gpg precisely has valid signature and good signature to distinguish signatures that are technically valid from good signatures.

e.g. a valid signature can be expired, so it's not good or shit like that.

So, the argument is flawed.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
That's no different from the apt-key adv --recv-keys <key id> that people do now on signature failures, so I don't see that as a reasonable argument.

We can only show the key id in the output, though, and then people have to go search on the web to get the full key (or they echo $signature | base64 -d | head -c32 | base64; but really, who does that).

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
Being able to validate consistency of a repository without needing additional files is a worthwhile goal. We also validate MD5 sums of hashes when downloading, despite them not being trusted (so they can only reject on failure). If there is an invalid signature, it should fail verification whether we trust it or not.

The other goal of the design - with the subkeys - is to be able to start signing repositories with a new subkey, without having to distribute the new subkey to users out-of-band.

The talk is unfortunately not understandable due to inadequate audio quality. It's also not the right medium to bring this across, you'd want to have a paper of up to 3 pages.

gpg has the same distinction between valid signatures and good signatures, though you need to distribute the key out of band there.

I don't see how this can confuse people. Nobody else should be implementing this specification.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
That's not how it works, no. This is a gpg specific issue as it does not communicate the key size over the status fd.

We don't parse the keys ourselves for obvious reasons.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
Still have to trust 1k RSA keys because GPG does not allow us to say they're unsafe.
julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
Please note that the spec is ahead of the implementation, and the project has been stale for 4 months. For spec and implementation to stay in sync, they'd have to be in the same repo I guess. For now, the spec is driving and the implementation catching up with it.

For more background, see

https://mastodon.social/@juliank/105679564085465156

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
For reference why I don't follow, the two word approach to determine the algorithm and then be able to parse the base64 directly into a struct are easy.

like in C++, I can

std::string algo, encoded;

in >> algo >> encoded;

if (algo == ed25519) base64_decode(encoded, ed25519_pointer); return std::move(ed25519_pointer); else if (algo == ed448) base64_decode(encoded, ed448_pointer); return std::move(ed448_pointer);

Where the _pointer are std::unique_ptr of Ed25519Signature and Ed448Signature which are each subclasses of Signature, and the return type hence std::unique_ptr<Signature>.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
> # The algorithm is stored with the signature. > This is slightly less bad than above, but still bad

So I just also realized that for the subkey scheme, it matters even less, because the algorithm is effectively part of the subkey packet.

It's not encoded in there, but how high is the chance that a different algorithm would produce the same signature for it?

We really just need the algorithm field to be able to determine how to parse that, but it reinforces itself by virtue of the subkey being signed by the primary key itself.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
That's an absurd statement. OpenPGP _is_ the legacy cruft. It's made for _emails_. It makes no sense elsewhere. The encoding for clearsigned signatures is overly complex.

And Rust is inappropriate in any case for now, it's not allowed in Ubuntu main.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
Yes, the implementation lacks behind a bit. Project's been stale a few months
julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
> They do say "there may not be any practical ed448 implementations", however the cryptography.hazmat.primitives.asymmetric library used by the reference implementation has an ed448 function that could very likely slide right in. They would however need to change some hardcoded lengths.

But that's not what that sentence means at all. It meant that APT might start just with support for ed25519, as IIRC I was told that ed448 is overkill right now.

I successfully have signed and verified signatures using Ed448 using the branch of python-aptsign for it.

> I do feel their discussion about not using separators to avoid coding string splitting actually makes it more complicated to code around variable lengths.

I don't follow

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
> It uses ed25519 but doesn't do anything about inconsistent validation criteria ( https://hdevalence.ca/blog/2020-10-04-its-25519am ), different implementations consider non-overlapping sets of signatures valid. If you want to have automated klaxons on signature failures, this ought to be addressed for the same reason as "repositories do not fail for a subset of users". (though it's not fatal in this application as it is in consensus systems).

This is annoying, but ultimately I guess not super relevant, as it needs to verify with APT, and that's about it in terms of requirements. And APT doesn't change it's crypto backend all the time, and we don't need to care about alternative implementations. It does mean we can't switch the crypto library inside APT I guess.

Does ed448 have the same issue? Do we have hardware tokens that support ed448?

The C++ code right now uses libnettle to verify signatures for the PoC.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
I don't understand your point about the public key. The goal is to be able to verify all signatures; whether we trust a signature or not is configured inside APT by specifying a list of trusted primary keys.

> # The algorithm is stored with the signature. > This is slightly less bad than above, but still bad

Which algorithms are trusted is a client side decision. We do need the string to be able to parse the signature itself. signify also has it, except that it includes it inside the base64 as the first two bytes.

I do not believe there is a need to be able to configure the trusted algorithms on a per primary key basis (aka storing it with the public key).

Basically the problem we have with GPG is that it trusts algorithms for far too long. So we had to implement lists of trusted algorithms inside APT ourselves. We still have no way to reject 1k RSA keys, though, because GPG does not tell us the key size.

We can assign algorithms trust levels and prevent downgrades to weaker keys alongside the other downgrade attack checks for Release files we already have in APT.

> I didn't look at the sub-key protocol in detail. The ephemeral key for every release is an interesting choice. The root key is "offline". But if it must be brought online to sign a new ephemeral key for every release anyway, you might as well just use it to sign the release itself.

Yes, well, if you go with the subkey approach you'll always have a subkey, so you can't sign directly with a primary key.

I think there's some confusion here in that people believe both formats should be supported at the same time, but they are two separate proposals, and only one should make it into real code.

> Using minisign/signify like OpenBSD does and keeping things very simple makes sense to me. The complexity designed into this system (sub-keys, multiple algorithms and signatures) starts to stretch the bounds to where TUF (https://theupdateframework.io/) might make sense. TUF is very complex and not worth it for most projects, but Debian is exactly what TUF is designed for.

I've said it 4 years ago - we don't want TUF. TUF does not provide useful things, but adds (duplicates) a lot of complexity. It's the opposite of what we're trying to achieve here.

https://blog.jak-linux.org/2017/08/17/why-tuf-does-not-shine...

Add to that that we'd have to rewrite the whole thing to use YAML/deb822 instead of JSON files for repository format compliance.

julian-klode··on Ditching OpenPGP, a new approach to signing APT repositories
> The lack of domain sep means stuff like a subkey itself is a valid payload signed by the master key

Well. There are two proposals. If we go with the subkey proposal, we'd not implement the simple protocol, solving that issue.

> Signer is setup to have the whole input in memory at once

It's fine.

> This text says very little about validating trusted keys. A signature where you just take the pubkey from the input

That would be silly. APT would maintain a list of allowed public primary keys. The spec hints at that with "Clients shall verify all signatures from supported algorithms in the Release file, whether a given key is considered trusted or not."

> No obvious affordance for additional signed metadata, which is useful for forensics. So, for example, if a project has multiple parties with access to the keys they may want to tag which user(s) authorized the signature or included a timestamp. If their signing is performed by a HSM (e.g. something like USB armory that allows no user access) the consistency of their metadata could be preserved. This could probably be kludged on by requiring the metadata just be inside the payload somewhere.

We don't have a use for that. The feature set of GPG is already over the top.

> It's not entirely clear to me that it's supported to hand out multiple subkeys to different parties all with the same serial, and they stay all equally valid until a key is observed with a newer serial (at which point all subkeys need to be updated)

Yeah, the serial exists to revoke all priorily issued subkeys, effectively.

julian-klode··on Apt 2.0
I know that, but compression time is not as relevant as you think it is. If you look back at the measurements we did 2 years ago:

https://lists.ubuntu.com/archives/ubuntu-devel/2018-March/04...

You can see that the performance gain from zstd under realistic scenarios is about 10%.

you can see that switching from xz to zstd only improved firefox install time from 37s to 33s; running in eatmydata to avoid fsync from dpkg improved it to 12.5s (8.5s with zstd).

I don't know what Arch linux does, but they likely do not correctly fsync() data after its been written to a temporary file, and then fsync the directory after files have been renamed to their final names, as is required to achieve consistent results in a crash.

julian-klode··on Apt 2.0
It literally is so simple you could do it by hand.
julian-klode··on Apt 2.0
What I do like is that, I think dnf, they do not switch your packages between repositories, even if you added a third-party repo with a higher version number.

But this is hard to do correctly for deb repositories. We first need to come up with a way to declare repository groups so you can say that e.g. the security repo can provide updates to the main repo.

julian-klode··on Apt 2.0
Like, zstd -21 --ultra requires a whopping 365MB of RAM to compress a 16 MiB file, zstd -19 requires 12 MB, and xz requires 10 MB.

Decompressing -21 requires 20MB vs 10MB for -19 and xz.

julian-klode··on Apt 2.0
What output? It downloads a file, then it sits around printing nothing for a few seconds (hooks are running), and then it updates it cache if necessary. Well, maybe it always rebuilds the cache, I'm not sure.
julian-klode··on Apt 2.0
We could reinstate wildcards (but not regexes, they are unsafe as their magic characters overlap with valid package names, so g++ can mean the package g++, the package g+, or any package matching the regex g+).

I feel like once people got used to patterns, that will be sufficient though, and the error reporting gets nicer.

julian-klode··on Apt 2.0
Non-parallel downloads from the same server are by design - turning that restriction off would be an easy thing. Server resources are limited, and people should not be cheating their way around bandwidth limits the server has. Put multiple mirrors in a mirrors.list, and then use `mirror+file://path to mirror list` instead of the http source in sources.list so that apt downloads from all these mirrors in parallel.

Be aware that high parallelisation of downloads may reduce your throughput for small number of things to fetch.

There also should be no latency need - APT keeps the number of requests it has sent to the server at 10, so there should be no latency overhead. Yes I know, this does not work for Google because their latency is crazy high, but their speed is super high too.

julian-klode··on Apt 2.0
It's ... tough
julian-klode··on Apt 2.0
Regular expressions are completely unsafe. See the clang++ example, which matches every package containing clang+ (that is 'clan' and one or more 'g'), unless of course somebody uploaded a clang++ or a clang+ package....

glob style wildcards are more common, though, in that context, anyway. It would be safe to re-enable those, but I'm not certain this provides a sensible user experience improvement.

julian-klode··on Apt 2.0
Last evaluation has shown that the size increase of zstd compared to xz warrants an introduction of delta debs first, so that people don't have to download more.

But then with deltas, mirror sizes grows even more, which is a hard sell.

Arch Linux gets away with using zstd -21 but that's not practical for general purpose Linux distributions, as it has significant memory requirements compared to xz.

Also the speedup in practice is negligent - while zstd is much faster than xz, most of the time installing packages is actually spent in fsync(). So you only see huge speedups when run in eatmydata.

← PreviousPage 2 of 4Next →