It doesn't look like WebTorrent is meant to deal with content that is continually changing, and I wasn't able to determine (after about 30 seconds of looking) how either of the two layers on top address that
It doesn't look like WebTorrent is meant to deal with content that is continually changing, and I wasn't able to determine (after about 30 seconds of looking) how either of the two layers on top address that
CacheP2P seems to solve the problem by including a list of sha1 hashes for all files that are distributed through P2P.
Speculation time:
I guess distributing the sha1 hashes for every wikipedia page is a bit too much (especially considering they are always changing). So with every page, you would only distribute the hash for those sites that are directly linked from it.
If you provide the correct sha1 hash for every linked website (which in turn includes the hashes of all linked websites from it...), you would form some kind of merkle-tree of wikipedia pages. Except that the graph of wikipedia pages is certainly highly cyclic (thus no tree), so this isn't feasible. And all the pages change all the time. So this naive approach has already two strikes against it and does certainly not work.
I will try to have a look at the source code at some later point, I guess.
5 Million articles, 512b per Hash, if that's enough, gives 2.5 Gb. Updates can be much smaller. Multiply that by 10 for hashes of the nearest users, who have a copy, and that's still manageable. Just of the top of my head.
edit: inb4 a blockchained approach.
5 million articles times 64 bytes per hash (512 bits) divided by 1e6 bytes per MB is 320.