You're right. It is very hard, if not impossible, to get absolute guarantees. Having said that, FPGAs can make supply chain attacks harder. See my other comments in this thread.
2,639 karma · joined June 16, 2012
Obsessed with building more trustworthy systems.
You are right. Apple is fully in control of the servers and the software, and there is no way for a customer to verify Apple's claims. Nevertheless system transparency is a useful concept. It can effectively reduce the number of things you have to blindly trust to a short and explicit list. Conversely it forces the operator, in this case Apple, to explicitly lie. As others have pointed out, that is quite a business risk.
As for transparency logs, it is an amazing technology which I can highly recommend you take a look at in case you don't know what it is or how it works. Check out transparency.dev or the project I'm involved in, sigsum.org.
> they are almost guaranteed to be misusing that data
That is very unlikely because of the liability, as others have pointed out. They are making claims which the Apple PCC architecture helps make falsifiable.
We don't, but using an FPGA can make supply chain attacks harder.
Let's assume you have a chip design for a microcontroller and you do a tapeout, i.e. you have chips made. An attacker in your supply chain might attack your chip design before the design makes it to the fab, maybe the attacker is at the fab, or they change out the chips after you've placed them on your PCB.
If you use an FPGA, your customer could stress test the chip by configuring a variety of designs onto the FPGA. These designs should stress test timing, compute and memory at the very least. This requires the attacker's chip to perform at least as well as the FPGA you're using, while still having the same footprint. An attacker might stack the real FPGA die on top of the attacker's die, but such an attack is much easier to detect than a few malicious gates on a die. As for covertly gathering or manipulating data, on an FPGA you can choose where to place your cores. That makes it harder for the attacker to predict where on the FPGA substrate they should place probes, or which gates to attack in order to attack your TRNG, or your master key memory. Those are just some examples.
If you're curious about this type of technology or line of thinking you can check out the website of one of my companies: tillitis.se
Thank you for bringing that up. Yes, it is an excellent example that proves the existence of silicon vulnerabilities that allow privilege escalation. Who knows whether it was left there intentionally or not, and if so by whom.
I was primarily arguing that (1) the technologies of Apple PCC are useful and (2) it is _very_ unlikely that Apple PCC is a ploy by Apple, to direct attention away from backdoors in the silicon.
FPGAs can help with this. They allow you to inspect the HDL, synthesize it and configure it onto the FPGA chip yourself. The FPGA chip is still proprietary, but by using an FPGA you are making certain supply chain attacks harder.
Great question. Most hardware projects I've seen that market themselves as open source hardware provide the schematic and PCB design, but still use ICs that are proprietary. One of my companies, Tillitis, uses an FPGA as the main IC, and we provide the hardware design configured on the FPGA. Still, the FPGA itself is proprietary.
Another aspect to consider is whether you can audit and modify the design artefacts with open source tooling. If the schematics and PCB design is stored in a proprietary format I'd say that's slightly less open source hardware than if the format was KiCad EDA, which is open source. Similarly, in order to configure the HDL onto the FPGA, do you need to use 50 GB of proprietary Xilinx tooling, or can you use open tools for synthesis, place-and-route, and configuration? That also affects the level of openness in my opinion.
We can ask similar questions of open source software. People who run a Linux distribution typically don't compile packages themselves. If those packages are not reproducible from source, in what sense is the binary open source? It seems we consider it to be open source software because someone we trust claimed it was built from open source code.
The technologies Apple PCC is using has real benefits and is most certainly not "all smoke and mirrors". Reproducible builds, remote attestation and transparency logging are individually useful, and the combination of them even more so.
As for the likelihood of Apple launching Apple PCC to redirect attention from backdoors in their silicon, that seems extremely unlikely. We can debate how unlikely, but there are many far more likely explanations. One is that Apple PCC is simply good business. It'll likely reduce security costs for Apple, and strengthen the perception that Apple respects users' privacy.
> when registers r0-rN are set to values [A, ..., N] and a jump to address 0xCONSTANT occurs
I would recommend something more deniable, or at the very least something that can't easily be replayed. Put a challenge-response in there, or attack the TRNG. It is trivial to make a stream of bytes appear random while actually being deterministic. Such an attack would be more deniable, while also allowing a passive network attacker to read all user data. No need to get code execution on the machines.
The benefits of transparent systems are likely considerable. The combination of reproducible builds, remote attestation and transparency logging allows trivial detection of a range of supply chain attacks. It can allow users to retroactively audit the source code of remote running systems. Yes, there are attacks that the threat model doesn't protect against. That doesn't mean it isn't immensely useful.
I hope you'll consider adding witness cosignatures on your transparency log though. :)
Would you be willing to clarify what you mean by that?
To start with, are you saying that Tor is wide open to traffic analysis
(1) by anyone,
(2) by powerful attackers such as the NSA, or
(3) by the NSA specifically?
Apple's Private Cloud Compute seems to be conceptually equivalent with System Transparency - an open-source software project my colleagues and I started six years ago.
I'm very much looking forward to more technical details. Should anyone at Apple see this, please feel free to reach out to me at stromberg@mullvad.net. I'd be more than happy to discuss our design, your design, and/or give you feedback.
Relevant links:
- https://mullvad.net/en/blog/system-transparency-future
- http://system-transparency.org (somewhat outdated)
Learn everything there is to learn about the Tillitis TKey. It's the most open-source software and hardware USB security token there is. It is FPGA-based, and contains a tiny RISC-V core.
Full disclosure: I'm involved in the project.
In any case we make the most important nuances clear on our landing page, and in other places on our website.
Two years ago we moved the OSS projects System Transparency and Sigsum to its own organization: Glasklar Teknik AB (glasklarteknik.se).
The OSS and OSHW project TKey was moved to Mullvad's other sister company Tillitis AB (tillitis.se)
The paragraph above is clearly visible on our landing page. We don't want people using our service for things it's not designed for.
The paragraph below is also a direct quote from our website.
"When you visit a website, you can be identified and tracked through your IP address, third-party cookies, all kinds of tracking scripts, and through so called browser fingerprints. That’s why masking your IP address is not enough to stop the data collection. However, by using a trustworthy VPN in combination with a privacy-focused browser, you can put up a better resistance against the mass surveillance of today. That's why we partnered with the Tor Project to develop Mullvad Browser – a browser designed to minimize tracking and fingerprints."
A VPN is not enough for privacy. But in combination with a privacy-focused browser, you make sure to block third-party cookies and other tracking technologies used by the data collectors.
That's why we partnered with the Tor Project to develop Mullvad Browser – a browser designed to minimize tracking and fingerprints.
Please also note that this information is clearly displayed on our landing page. We don't want people using our service for things it's not designed for.
I should've been more clear. The questions I posed above are rhetorical. I've spent well over half a decade obsessing over them. See my mention of System Transparency, Sigsum and Tillitis elsewhere in this thread.
I can't speak for my marketing colleagues but I would assume they reason about it. It seems like a complex system to me, which means the approach kind of has to be (1) forming an understanding of the system and (2) deciding whether one is comfortable with the uncertainty.
One aspect that makes the cost/benefit assessment quite a bit easier is that we don't only care about how many new paying customers we get. It's also fun to do this kind of advertisement. How many interesting conversations about privacy have been had as a result of people seeing our billboards? That's worth something too.
I'm sorry to hear that. For what it's worth our marketing colleagues make a big effort to minimize the risk of such interpretations. Sometimes a really snappy string of words can be interpreted multiple ways. There's also only so many words we can put on an ad before it gets messy. We do try hard to make the nuances clear on our website, which ultimately is where any new users will have to go in order to buy the service.
- https://www.sigsum.org (a transparency log with witness cosigning)
- https://tillitis.se (an open-source hardware FPGA-based security key with measured boot)
Yes. It is quite an interesting situation, really. It's also a fun challenge! To what extent can we prove that we are trustworthy, and using what tools? Do those tools exist or do we have to invent them?
1. We launched Mullvad 15 years ago. During those 15 years people's interest and awareness of online security and privacy has grown considerably, as has the consumer VPN market.
2. Our strategy is quite different from most of our competitors'. As a result we've grown slower than several of them, but we have nevertheless continued to grow year after year.
3. The costs of the campaign are perhaps lower than you assume.
tl;dr We've slowly grown over many years and are now making enough money to plaster privacy propaganda over your city. Hopefully it's an interesting change from the usual bus ads. Enjoy!
On the other hand the campaign we did in Stockholm last year worked out quite well. It managed to affect both domestic and EU legislative discussions at the time. Or at least our campaign contributed to moving the discussion in the right direction.
How much is that worth? I'm not sure, but the reason we started Mullvad in the first place was to conduct political action through entrepreneurship, specifically regarding mass surveillance and censorship.
If nothing else it seems to amuse a lot of people, including me and my colleagues. When I first heard of the idea of plastering privacy propaganda all over some major U.S. cities my initial reaction was more or less "lol, we can just do that?". As it turns out we can. :)
As you point out yourself the trust assumptions are fundamentally different. As a result a transparency log can produce an inclusion proof and a consistency proof that is on the order of a few kilobytes large. A trust policy for such a verifier would be of similar size.
How large would the equivalent inclusion- and consistency proofs for a proof-of-work-based blockchain be? What would the trust policy look like? Presumably the verifier would use some historical block as a starting point, which would constitute the trust policy. It would then need to process the entire blockchain from that point and onwards up until the point of inclusion. That's way more than a few kilobytes.
Again, I haven't paid attention to blockchains for a long time so there might be some consensus mechanism out there that has solved this somehow. I'd love to be proven wrong.
The design you're describing doesn't have those security properties.
A malicious or compromised Log conducts a split-view attack by showing different views of the Log to different parties. This is done by providing different parties with different inclusion proofs and consistency proofs. Thus the Log remains consistent from the respective network vantage points of the target(s). They all see an append-only log.
A Client or Monitor shouldn't merely be concerned with whether the Log is consistent from their vantage point. It should be concerned with whether the Log is globally consistent, i.e. whether all views of the Log are consistent with each other. Historically this has led to designs where the Clients gossip consistency information to each other, the introduction of Log Auditors and witness cosigning. I prefer witness cosigning, although to be fair Clients gossiping can work in some designs. Apple's transparency log design is one example, if I remember correctly. It's been a while since I thought about transparency logs.
The fundamental value of a transparency log is to provide trivial discoverability. Say an attacker has compromised a code signing key and is using it to sign a malicious software update. Let's also assume that the attacker can distribute this update to a client that will trust it. Such a targeted attack is not trivially discoverable by the software vendor if the client's trust policy only requires a code signing signature by the compromised key. If the client's trust policy were to also require a proof that the signature is included in a trusted transparency log, the software vendor can discover the attack if it monitors the log. Note that transparency logs doesn't directly prevent attacks - it makes them discoverable.
Transparency systems based on transparency logs have five components: Log, Submitter, Client, Monitor and Auditor or Witness(es). The Log is a Merkle tree with its current root tree hash signed. In the example above the Client would be the software updating mechanism, which has a given Log in its trust policy. The Client will only trust a software update from the Submitter if the Submitter's signature and the update's hash are included in the Log. The Monitor regularly polls the Log for new entries it cares about, in this case software updates signed by the Submitter's public key. The Auditor is another party that regularly polls the Log, but the Auditor's job is not to discover log entries that constitute an attack. Instead, the Auditor's job is to monitor the Log's consistency - i.e. whether the log has removed any previously added entries. This is called a split-view attack, and it is a concern in all widely deployed transparency logs today. An alternative to Auditors is cosigning Witnesses, which proactively witness a Log's consistency, and whose witness signatures are required by the Client in order for it to trust the Log. This results in a much stronger threat model. See sigsum.org for more details on such a design.
Getting back to your comment:
- Yes, Clients are "thin", to use blockchain terminology, but they don't monitor the Log in order to detect attacks. The Log is part of their trust policy for making some trust decision, such as trusting a software update. Monitoring the Log for new entries that indicate e.g. key compromise is the Monitor's job.
- Regarding accountability: The Client's trust policy in conjunction with the Log and the Monitor increases the Submitter's accountability to the Client. Similarly the Auditors or Witnesses increases the Log's accountability to Clients, Submitter and Monitors.
- Regarding inclusion in the Log (or blockchain) I'd use the word "authority" as in who has the power to prevent something from being included. As you point out a transparency log is a central entity which has the sole power to decide what gets included or not. If the log is shut down, or its public key destroyed, nor more inclusions are possible. In this sense a blockchain is far more censorship-resistant than a transparency log. In other words, it is important that Clients trust multiple logs for redundancy. The consensus mechanism of a transparencty log is simply a public-key signature, as opposed to an emergent authority in the form of e.g. a distributed timestamp server based on SHA256. This vast difference in how the database's integrity is defined - the consensus mechanism - is why blockchains work better than transparency logs for censorship-resistant digital currency, whereas transparency logs work better than blockchains for efficient inclusion- and consistency proofs.
Well said!
Imagine an embedded system that only accepts firmware updates that are digitally signed and discoverable in a public append-only log. Such a proof would have to be attached to the firmware update just like the digital signature, with a size constraint on the order of kilobytes. To be fair I haven't kept up-to-date with blockchains, but I find it hard to believe that any existing blockchain could solve this. Transparency logs can.
I wonder when businesses that currently use blockchains as immutable logs will realise that there is a more cost-effective solution. One of the reasons it hasn't happened yet is because blockchains have magnitudes more mindshare than transparency logging. In fact, there doesn't seem to be that many people who deeply understand the design parameter space of transparency logs.
Shameless plug: My colleagues and I have spent a few years designing one - Sigsum. As Filippo mentions he has contributed to its design as well.
A prioritised list of the most important cryptographic constructions may very well look like this: symmetric ciphers, hash functions, MACs, public-key cryptography, transparency logging.
If you're interested in security and/or cryptography and you don't know what transparency logging is, you're missing out.