ZeroNet – Uncensorable websites using Bitcoin crypto and BitTorrent network
zeronet.io
zeronet.io
Little known fact: A single miner has close to 65% or more mining power on Namecoin. Reported in this USENIX ATC'16 paper: https://www.usenix.org/node/196209. Due to this reason some other projects have stopped using Namecoin.
I'm curious what the ZeroNet developers think about this issue and how has their experience been so far with Namecoin.
.com, .net, .org domains are handled differently, and may be easier to lose permanently.
It seems like a publisher-addressable network (where documents are identified using a publisher's public key) or a content-addressable network (where documents are identified using a file hash) would be good enough by itself, so long as the protocol had builtin support for distributed document searching and ranking.
Casual internet users on the regular internet do not seem to be using domain names to locate resources anymore. They are using Google to locate resources, and only looking at the domain name to verify identity. If the primary purpose of the domain name is not to locate a resource but to verify identity, then it seems like this could be accomplished with a web of trust without a central name registrar.
* 2 years out of date gevent-websocket
* Year old Python-RSA, which included some worrying security bugs in that time. [0](Vulnerable to side-channel attacks on decryption and signing.)
* PyElliptic is both out of date, and actually an unmaintained library. But it's okay, it's just the OpenSSL library!
* 2 years out of date Pybitcointools, with just a few bug fixes around confirmation things are actually signed correctly.
* A year out of date pyasn1, which is the type library. Not as big a deal, but covers some constraint verification bugs. [1]
* opensslVerify is actually up to date! That's new! And exciting!
* CoffeeScript is a few versions out of date. 1.10 vs the current 1.12, which includes moving away from methods deprecated in NodeJS, problems with managing paths under Windows and compiler enhancements. Not as big a deal, but something that shouldn't be happening.
Then of course, we have the open issues that should be high on the security scope, but don't get a lot of attention.
Like:
* Disable insecure SSL cryptos [3]
* Signing fail if Thumbs.db exist [4]
* ZeroNet fails to notice broken Tor hidden services connection [5]
* ZeroNet returns 500 server error when received truncated referrer [6] (XSS issues)
* port TorManager.py to python-stem [7] i.e. Stop using out of date, unsupported libraries.
I gave up investigating at this point. Doubtless there's more to find.
As long as:
a) The author/s continues to use out-dated, unsupported libraries by directly copying them into the git repository, rather than using any sort of package management.
b) The author/s continue to simply pass security problems on to the end user
... ZeroNet is unfit for use.
As simple as that.
People have tried to help. I tried to help before the project got as expansive as it is.
But then, and now, there is little or no interest in actually fixing the problems.
ZeroNet is an interesting idea, implemented poorly.
[0] https://github.com/sybrenstuvel/python-rsa/issues/19
[1] https://github.com/etingof/pyasn1/issues/20
[3] https://github.com/HelloZeroNet/ZeroNet/issues/830
[4] https://github.com/HelloZeroNet/ZeroNet/issues/796
[5] https://github.com/HelloZeroNet/ZeroNet/issues/794
It's a shame your skills weren't more appreciated.
My problem is conversations like this one: [0], where improvements are resisted against, for being too hard.
People have tried to help improve quality and testing rigour, but they get turned away.
Nobody is going to attack ZeroNet if it doesn't have users anyway.
> I wasn't aware of any hackers. The only problem I have since I have been running ZeroNet for a year, is the minor problem of file size mismatch, simply because not all peers in the network have the latest version of a file.
At best, that's an unhelpful attitude. It leads to things like: [0]
[0] https://arstechnica.com/security/2017/03/firefox-gets-compla...
It's easy to point issues and not do anything to help.
We talked it over, decided I would do the test suite.
I started, found the bad practices, and showed how I could turn it into a fully automated system, new versions could be tested against, and if it works, it could output binaries for every system.
The response was, 'No don't do that. I like doing it manually. Means I can check for breakage.'
Followed by my PRs and issues being closed, and my emails bouncing.
Can you take a look at it again? It's not my area of expertise.
About 52% test coverage, and pip is in use for some things.
However, so long as the LIB[0] folder exists, these sorts of problems will recur.
Each of those libraries is an opportunity for problems to emerge.
However, as they're manually managed, you don't get the chance to test against future versions, to check for breakage or okays.
Out of date becomes inevitable.
[0] https://github.com/HelloZeroNet/ZeroNet/tree/master/src/lib
*I am referring to concentrated power of the big players here, country-wide firewalls, and bureaucracy towards how/what we use.
https://www.ribbonfarm.com/2010/07/26/a-big-little-idea-call...
ZeroNet does appear to take an important step in that direction with decentralization.
Well. Look, even if you have multiple internets, decentralized everything, distributed all systems, no more Google no more Facebook. What does the communication patterns in such a system look like? Do you use the system after work before going to sleep? Your and everyone else usage patterns, traffic can be analyzed. The endpoints, many of them would be honeypots run by Spooks, revealing even more what you are up to and giving you a false sense of safety while the Spooks could run the entire decentralized inter-network.
So your system would have to fake it somehow, fake requesting some hashes here and there, fake request/post comments and follows. Otherwise, the social data available when the SPOOKS join your social-network even if it is distributed like patchwork on scuttlebot, defeats its purpose.
That is what bitmessage does, but then you pay in high bandwidth costs. And yet, you cant just do random shit, random can easily be filtered out so you need more advanced method of finding fake social relations and using those to do fake data to actually conceal what you and everyone else is doing on the interweb.
EDIT: Im not saying "give up", its a very worthy cause, just the problem is harder and enters the social space quite fast - the problem is the same as "we are all nice developers and hackers" yet 99% seem to be employed by NSA/similar-services/Google and think they are doing great James Bond like type of jobs, while they are actually anti-hackers and anti-developers, in fact, anti-society.
ZeroNet is created for fast, dynamic websites, IPFS is more like a storage solution. It's already possible to cooperate, eg. IPFS for static, big files, ZeroNet for dynamic user content
Given the long history of vulnerabilities in the the browsers, trusting js from a well-known website might be OK, trusting js from zeronet is unreasonable.
If ZeroNet could run with js code generated only by the local daemon or without js it would be brilliant.
EDIT:
Found it :D
Browser support is noted here: http://caniuse.com/#feat=subresource-integrity
Another option would be to just use a subresource-integrity protected script to check the hash of a downloaded image/video before displaying it.
Frankly, given much of the history of successful Internet tools & protocols, I'd love to see some text-UI clients for ZeroNet.
Freenet: https://freenetproject.org/
ZeroNet uses the torrent protocol and serves up the content you've chosen to view. You know what you're serving.
ZeroNet doesn't really use the torrent protocol. It has its own file sharing service that it runs to receive requests from other users for files. It uses the torrent trackers to map site addresses to IP or onion address.
You know what you're serving initially but the site author can add any files they choose and you'll start serving them if you've visited the site and are seeding it. You have no control over malicious sites that decide to store arbitrary data.
Also, it's written in Java.
(Not sure if that's a praise or a criticism.)
As much as I dislike Java (ML outclasses it as a language), that's probably much better than the obvious alternatives (C and C++): there aren't nearly as many undefined behaviours, and that eliminates a whole class of potential security vulnerabilities.
If it were written now, Rust could be a viable alternative: just as safe, potentially faster, with less dependencies (at least as far as the compiled binaries are concerned).
Now as a language, I maintain that ML is better than Java on pretty much every account. It has sum types (or algebraic data types), a safer type system (without null), better generics (that weren't bolted on after the fact), a fine module system, easier lambdas… Ocaml in particular even has a class system, though it is almost never used: ML hardly ever need inheritance, so I count that as a negligible disadvantage. And of course, polymorphism (the OO kind) is easily obtained with with modules or plain higher-order functions.
Yes, yes, Java has an enormous community, loads of tools, and so many libraries that whatever you want to do has probably already been done. Yes, yes, it means that many projects would be cheaper and faster to write in Java, thanks to those libraries, communities, and plain available programmers. The JVM itself is pretty amazing, with a tunable garbage collector, and very fast execution once the JIT has warmed up.
While important, none of those advantages come from the language. They come from Sun, the staggering amount of working Java devs, and the heap of work they already accomplished. Network effects, mostly. A similar comparison could be made between C++ and Rust, though I don't know Rust well enough to have an actual opinion.
---
Also, "you people" should also admit that a language can be better than another, even if it's only Java8 vs Java7, or C++11 vs C++98. You should also realise that it's important to distinguish the language from the rest (community, tooling, libraries). If you don't, the older language always wins! Of course you wouldn't start a sizeable project in Rust in a C++ shop. Throwing away all the internal libraries and framework, all the training and painstakingly acquired experience? Of course not.
Still, one must not lose the long term picture. At that picture is provided by the language. Because ultimately, everything stems from the syntax and semantics of the language.
or even that it's anything like being gay?
How does this track with the Tor Project's advice to avoid using BitTorrent over Tor [1]? I can imagine that a savvy project is developed with awareness of what the problems are and works around them, but I don't see it addressed.
[1] https://blog.torproject.org/blog/bittorrent-over-tor-isnt-go...
Though I guess unless you create a new identity for every site you want to post a comment on, your comments on one site could be proven to be posted by the same person as your comments on another site.
This project is about hosting generally. But if it were used for HD video streaming, Tor Project would be just as unhappy.
That's a privacy concern, not a load problem.
I think it's fairly clear at this point that ZeroNet isn't testing to make sure that this is the case.
Their TorManager [0] is basically a wrapper around the tor executable, and runs a fairly vanilla config.
So yes, leaks or attacks via bittorrent are actually an issue here.
[0] https://github.com/HelloZeroNet/ZeroNet/blob/master/src/Tor/...
Its protocol is a different one.
https://zeronet.readthedocs.io/en/latest/help_zeronet/networ...
Also, I wonder if MPTCP would play nice with ZeroNet. MPTCP works very well With OnionCat. I could create TCP streams with hundreds of subflows over all possible combinations of multiple OnionCat addresses.
https://ipfs.io/ipfs/QmUDV2KHrAgs84oUc7z9zQmZ3whx1NB6YDPv8ZR...
https://ipfs.io/ipfs/QmSp8p6d3Gxxq1mCVG85jFHMax8pSBzdAyBL2jZ...
Which logically leads to an unrelated question -- if ISPs are doing DPI on every packet, they at least theoretically 'know' whether you're transmitting 'illegal' content. If I were a rights holder, I'd be making that argument against ISPs. I don't know how I'd sleep at night, maybe, but I wouldn't let ISPs have their cake (valuable user data) and eat it too (immunity based on status as ISP-only).
Unlikely.
To become a peer, you must first visit the website, fully downloading the content.
Which makes an argument to you consenting to share the information.
Governments do and have asked ISPs to update and block against certain websites. Thus, the automated system is expected to behave adequately.
However, as a person becomes involved, they become an active participant in the pipeline.
That is a huge theoretical difference.
A person is not automated, they have common reason, and intellect that goes beyond the rules that can be encoded in a dumb system.
This reasoning is what allows us to hold a person accountable to their actions.
If you visited a ZeroNet site, found it was bad stuff, immediately left and deleted the cache, you might have a case for innocence.
But if you immediately left, but continued to actively share the content... It's a different message.
You could become part of a child pornography ring, for example.
And courts enjoy making examples of distributors of such materials.
Pleading your innocence, becomes difficult at that point.
You've shared illegal content from your own property.
Either way it's a damn difficult question to answer, but God's be damned if I wouldn't prefer a distributed internet.
At least IPFS is working hard towards Tor integration. That might be something one day.
Actually, that day is today already! OpenBazaar had the same need of a Tor transport and made one! It's available here: https://github.com/OpenBazaar/go-onion-transport/
Basically a plug-and-play transport for IPFS.
- https://github.com/OpenBazaar/openbazaar-go/blob/4a9ee8de8fd...
- https://github.com/OpenBazaar/openbazaar-go/blob/4a9ee8de8fd...
Hope that helps a bit. Keep in mind that none of this have been verified and might not work as advertised. Just a warning.
The ISP, like a container ship, is not responsible of every bit it moves around. Individuals are.
https://www.eff.org/pages/playpen-cases-frequently-asked-que...
However, any Whonix users would not have been affected, for two reasons. One, this was Windows malware, and Whonix is based on Debian. Two, Whonix comprises a pair of Debian VMs, a Tor-gateway VM and a workstation VM. Even if the malware had pwned the workstation VM, there is no route to the Internet except through Tor.
What works is using Tor.
to this:
What works is using [Whonix].
Is there some network isolation going on which prevents that?
Is this some kind of 'vm specific' virtual network which can't talk on the real lan? Is that implemented on the hypervisor?
Edit: I forget that I'm writing on HN. When I say VM, I'm referring to full OS-level VMs, not namespace, Java, etc VMs.
That's a bit of a nonstarter for a few of.
We probably aren't the target base for the project though so maybe it doesn't matter...
Years ago, I created a LiveDVD with VirtualBox plus Whonix gateway and workstation VMs. I had to hack at both Whonix VMs to reduce size and RAM requirements. But I got a LiveDVD that would run with 8GB RAM. It took maybe 20 minutes to boot, but was quite responsive.
You must isolate Tor process and userland in separate VMs, or even separate physical devices. Even if the browser gets pwned, and the NIT gets dropped, you'll be OK, because the Internet is reachable only through Tor. Whonix is an easy to use implementation.
I've been ragging on Tor Project about this for years. But they don't want to frighten people by making Tor too complicated to use. You could be cynical, and say that they want the cannon fodder for their government masters. Or you could say that they think it's more important to protect the most people, rather than to most strongly protect technically competent people. I have no clue what the truth is. Maybe there's a range of opinion.
But Whonix really is trivial. You install VirtualBox. You download the Whonix gateway and workstation appliances. You import them in VirtualBox. You start them. You work in the workstation VM. There's nothing to configure. That literally should be enough information to use Whonix. Plus there's a wiki and a support forum.
I read up on this a while ago, but didn't keep links. There was some discussion on /r/Freenet. For example: https://www.reddit.com/r/Freenet/comments/5tnx81/freenet_use... Missouri police developed a custom Freenet client that logged everything. But I don't remember the name :(
I should have said: "What works is using Whonix, or otherwise using Tor securely with leaks blocked."
I haven't looked deep into any of these projects, but I do think they are neat and hoping at least one of them gains a lot of traction.
I cannot help but feel disappointed and unamused.
Oh, they say the following on their issue tracker:
> zeronet protocol is different from torrent, so libtorrent will not work.
So the headline is misleading?
[1]: https://docs.google.com/presentation/d/1_2qK1IuOKJ51pgBvllZ9...
Huh? What do they mean?
If you haven't previously visited a page then the response time will be limited by how many peers are available <b>and then</b> by your connection speed.
[0] - https://docs.google.com/presentation/d/1_2qK1IuOKJ51pgBvllZ9...
Then you can have users end up browsing stale versions of the site. Still an issue as of before Christmas last year.
ZeroNet is a application for distributing P2P applications, using Bittorrent for the P2P layer. In theory, ZeroNet could leverage IPFS to get a better and more modular stack for the actual connectivity and transfering.
That might be possible, but with the prvelence of magnet links instead of everyone using that, I just assumed not.
# create empty folder
FOLDER=`ipfs object new unixfs-dir`
# add folders to object
FOLDER=`ipfs object patch "$FOLDER" add-link 2017-01 "$HASH_2017_01"`
FOLDER=`ipfs object patch "$FOLDER" add-link 2017-02 "$HASH_2017_02"`
FOLDER=`ipfs object patch "$FOLDER" add-link 2017-03 "$HASH_2017_03"`
# print final hash
echo $FOLDER
This would result in a hash that has the folders for January, February and March, which can have arbitrary nested folders and files in them.But that'll need revising once sha1 torrents are deprecated.
Not torrents themselves, only torrent search engines. Torrents are distributed by design, but traditional torrent directories/aggregators/search engines are centralized, thus easy targets for DCMA take-downs, ISP blocks, trials etc.
ZeroNet is perhaps not enough on its own to solve this problem, though, since a good torrent search engine suffers from the same limitation as a good forum, which is the need to have some form of community-based moderation. If people can't remove spam search results, and spam comments, then the medium can be too easily exploited (using Sybil attacks, etc.) and become useless.
The missing piece which is holding back so many decentralised technology projects is a lack of a decentralised trust platform. A necessary step towards this would be a decentralised (and privacy-preserving) identity platform, which would have the added benefit of removing the "Log in with Facebook/Google" problem from the web.
2) Does filestore also let you store, say, newly pinned files in your regular file tree? That is, can you pin a hash for a file (or tree) you don't already have and provide an ordinary file system location where it should go when it's downloaded? Or do you have to copy it out of IPFS' normal repo manually, then re-add it in the new location? Also: how does filestore behave if files are moved/deleted?
3) What rate of repo changes requiring upgrades can we expect for the future? That is, how stable is the current repo structure expected to be? Is the upgrade process expected to improve and/or become automated any time soon?
4) Is there a table of typical resource requirements somewhere? I'm looking for "if you want to host 10TB and a few 10s of thousands of files, you need a machine X GB of memory. If you want to host 500MB, you only need Y GB of memory. If you have 2TB but it's in many, many small files, you need Z GB of memory", or else a formula for a achieving a best-guess for that. For that matter, how predictable is that at this point?
The use case I've been excited to use IPFS for since I found out about it is a private, distributed filesystem for my friends and family. Easy automated distributed backups/integrity checking on multiple operating systems, access your files at someone else's house easily, that sort of thing. Filestore finally landed, which was a big piece of the puzzle (the files have to remain accessible to ordinary tools and programs or I'll never get buy-in from anyone else), so that's exciting. Now I'm just waiting for docs to improve (so I'm not searching through issue notes to learn which features exist and how to use them) and for a sense that it's stable enough that I won't be fixing brokenness on everyone's nodes several times a year.
2) The latter. Former is nice idea you for sure should rise it on go-ipfs tracker.
3) The repo update is currently automated (run daemon with `--migrate` flag so it will migrate itself)
4) Unfortunately not but it is very interesting question. If you could ask it on http://ipfs.trydiscourse.com/ it would be awesome.
[1] hashrates can't be compared directly due to different hashing algorithms having different costs for producing a hash.
But this guy said that a single miner has 65% of the hashing power: https://news.ycombinator.com/item?id=14043038