IPFS and Their Gateways
daniel.haxx.se
daniel.haxx.se
> the gateway is in full control and can inspect and tamper with the data as much as it likes. And there is no way for the client to know or detect if it is happening.
Regarding data tampering;
IPFS uses content based addressing, the point is that the client _can_ know by computing a message digest of the incoming data. The verification is fast and cheap.
And yet the most widely used IPFS client, Brave, does not implement this.
The blog post is technically wrong, but I think it's worth accounting for the general state of the ecosystem in these decisions, and in whether to include IPFS in cURL, and it sounds like the general state of the ecosystem is not up to the standards that cURL should reasonably expect from a protocol asking for inclusion.
You only have to verify the content yourself if it is not being verified by someone you trust (i.e. a local gateway that you're running yourself).
For example, debian-10.7.0-amd64-netinst.iso has SHA256 checksum b317d87b0a3d5b568f48a92dcabfc4bc51fe58d9f67ca13b013f1b8329d1306d. Here are two example CIDs generated from that file:
https://cid.ipfs.tech/#bafybeihjy54iyvheotna2aeqmzhqnro6yot4...
https://cid.ipfs.tech/#bafybeihfqpypuhmtyzazrj3g4b4f4nqk2ziy...
Notice that neither one contains the original checksum.
Here is IPFS's master issue for "verifiable gateway responses", which summarizes the technical challenges and progress: https://github.com/ipfs/in-web-browsers/issues/128.
It needs censorship-resistance added. It needs user privacy added.
Both of those could be enabled by combining data blocks semi-randomly. Ie. rather than downloading the data you're looking for, you download a block of data containing what you're looking for and some other data. Then nobody knows what it was you were looking at. Some combined blocks might contain a few bytes of data from thousands of files.
For censorship resistance, rather than users pinning individual files, the network needs to assign them blocks they must store (or be kicked off the network). Some of those blocks will be combined blocks which contain a few bytes some users may not wish to store - and that's fine, it means IPFS isn't for them.
Remember that data can be encrypted by content hash, so there is no way to know what you're storing unless you go scouring the public internet for a key to decrypt it.
http and tcp, other transport systems, also do not have censorship resistance. it's the wrong place for it.
you can use ipfs to build a censorship resistant system, just like you can with tcp, but that is the purview of a higher layer.
I don't think it's possible to reasonably to maintain layering while adding privacy and censorship resistance.
That IPFS does NOT put data I did not explicitly requested on my storage is exactly why I feel comfortable using it compared to some other distributed storage systems where I might end up with horrific data on my system.
And encrypted data you don't have the key for is indistinguishable from random noise. 'Combined' blocks that you don't know which partner blocks are necessary for decoding are likewise useless.
You're being like a mailman handling mail - you can't look into the mail, so you can't know if you're delivering love letters, junkmail, or terrorist propaganda.
ISPs and USPS have common carrier protection, by default a home user does not.
The underlying problem here is that it’s basically impossible to distinguish between someone naively allowing strangers to use their property and someone who is trying to cover their tracks by pretending to be unaware, and the internet is full of people who will abuse any free service. If it happens once, maybe you can plead ignorance but that’s not a foundation for a robust file sharing network.
The danger is just too big, as my sibling commenter says.
If Uber Package driver delivers a package of drugs - pretty sure he won't get to jail as it will be trivial to prove he didn't put is there and had no idea.
In your mailman example, it would be like the FBI announcing exactly what the envelope looks like, uniquely identifiable, and if you encounter this letter to report it immediately or else your complicit. Or at least, that's the fear.
Yes, if it's encrypted you have no idea, but if anyone ever found out what was in it you can know who ever transmitted it. Unless everyone re-encrypts it repeatedly.. which then makes content address-ability awkward.
This sentiment also applies to your argument, i.e. IPFS isn't fit for your purpose; and that's fine, it means IPFS isn't for you.
There are a whole bunch of censorship-resistant, plausibly-deniable systems out there (e.g. I run FreeNet from time to time)
What you're looking for is Oblivious Transfer or Mix Nets. Unfortunately, no practical solutions currently exist (except for Onion Routing).
> Remember that data can be encrypted by content hash, so there is no way to know what you're storing unless you go scouring the public internet for a key to decrypt it.
Censorship does not work this way. First, content that requires censorship resistance is usually public, otherwise one can employ a number of p2p or centralized solutions to share content directly.
Public content is easy to find, so step one: you find and identify the list of nodes that serve the content. Step two: you block the nodes on a level that's available to you (depends on the country). Step three: your users are unable to use the network effectively, because half of the nodes don't work, and revert to using Facebook.
Inspect, yes, but tamper? IPFS content IDs are hashes, so it seems like it should be possible to verify what you have been sent if you really cared enough to do so.
There isn't a tool today that can verify an IPFS address without actually downloading that file's DAG, which is basically downloading the entire file.
Imagine if I gave you a file out of a git repo, and the long commit hash. It's insufficient on its own to verify integrity, you need the intermediate nodes.
With ipfs you can infer that it used the reference chunker and try to reproduce the DAG but there is no guarantee it will work.
Better to just speak ipfs to the p2p net and get the DAG.
Git the Merkel root is a list of directories and hashs of whole files.
Ipfs, an individual file can be made up of many trees and those trees can have trees.
Example: https://explore.ipld.io/#/explore/QmbGtJg23skhvFmu9mJiePVByh...
That is a single file, but good luck verify that file without actually downloading the entire dag tree which includes the file.
Worse, that same file can be chucked and bucketed infinitely number of ways. So if I download that file and the re-add it to IPFS, I will very likely get a _different_ address. And that address will likely share _none_ of the of the nodes from the previous address. Meaning even though the same file is in IPFS, they are completely distinct and don't actually help seed each other.
https://blog.cloudflare.com/e2e-integrity/
It looks like they've taken the FF addon down and archived the source repo though.
And believe it or not - occasionally such content is used as a weapon. A Twitch streamer with an open Dropbox learned the hard way. It's used regularly on _that_ kind of imageboard to derail threads. I know because a naive version of me briefly ran an archive of a supposedly worksafe board about video games a decade ago and had their door knocked down at gunpoint about a year later.
I don't know how the Fediverse still exists, considering that most implementations are very eager to mirror all media they encounter.
I know HTTP can provoke the same caching, but for that there's actually much more robust legal precedent.
Also you can use the CID (which is a hash), to verify the content hasn't been tampered with.
People, even in this thread, keep spouting privacy.. when i'm not sure if i've ever seen that as a goal.
If anything content addressability makes privacy remarkably difficult.
With DoH generally must both opt in and must choose your provider.
0.004096 0.008192 1
0.008192 0.016384 3
0.016384 0.032768 2
0.032768 0.065536 121
0.065536 0.131072 156
0.131072 0.262144 205
0.262144 0.524288 304
0.524288 1.000000 98
1.000000 2.000000 141
2.000000 4.000000 24victorious-capuchin|> The main issues seem to be that IPFS is not yet up to the standards that cURL should reasonably expect from a protocol asking for inclusion. In particular, there are concerns about the lack of censorship resistance and user privacy.
a better analogy than http urls would be git commit hashes.
The point of IPFS is that the server can't tamper with the content because it would break the hash, but if you're using an untrusted proxy then it is just as bad as using an untrusted proxy for your normal web browsing.
For the web, TLS provides tamper-resistance. Accessing the web through a HTTP proxy would allow tampering, similarly to accessing IPFS through a gateway.
And said proxy could bounce your request around for max 30 hops before a random IPFS-proxy honors the request. This is horrible for user privacy.
Brave browser [1] at least has a native support for IPFS and no need for proxies at all.
> Other IPFS privacy concerns: Brave, the browser, has actual native IPFS support (not using any gateway) and they have an informative page called How does IPFS Impact my Privacy?
> By default, Brave will load the URI being requested via a public HTTP gateway; however, it will also show an infobar that asks you if you’d like to use a local node to resolve IPFS URIs.
So Brave by-default uses a public gateway.
It literally says: I have insisted, in the PR for ipfs support to curl, that the IPFS URL handling code should not automatically follow such gateway redirects, as I believe that adds even more risk to the user so if a user wants to allow this operation, it should be opt-in.