Asking for feedback on an idea: scatter storage across home computers | Hacker News Reader

Asking for feedback on an idea: scatter storage across home computers

3 pointsandrewstuart7 comments

Could this idea work? Maybe someone has built it already?

The idea is that you run a storage server application on your home computer.

You get paid for storing blocks of data which are encrypted.

The more reliable and fast your server is, the higher you get paid.

Customer of the system submit a file to be stored which gets broken into blocks and stored across many home storage servers. Customers can choose the amount of redundancy required -i.e. a block gets stored on 20 different servers across the world. More redundancy costs more.

Any thoughts on whether this sort of thing might be practical?

View on HN

Yes, this was a popular topic of discussion around 2001. The net result is that your home computer is so unreliable and has such little bandwidth that its spare resources are worth zero. Specifically, repairing data lost due to churn can easily consume all available bandwidth.

http://www.slideshare.net/emery/tfs-a-transparent-file-syste... (especially slides 21-23)

http://www.cs.umass.edu/~emery/pubs/TFS.pdf

http://tahoe-lafs.org/trac/tahoe-lafs

10 years down the track are things the same?

Worse. Systems are more likely to have an always-on connection than they were ten years ago, but the amount of upstream bandwidth has not increased proportional to the amount of data that users wish to store. Integrity checking and replication will eat you alive. The best place for you to start is the following paper from 2003 (check CiteSeer):

High Availability, Scalable Storage, Dynamic Peer Networks: Pick Two

If you can't figure out a way to get around the problems outlined in this paper then do not waste your time. A built-in incentive/payment system is not going to solve the problems you think it will and is really the start of another "and now you have two problems" koan. Trust me on this one...

BTW, the answer to your "Maybe someone has built it already?" question is that it was called Mojo Nation and as enticing as the concept may appear, it won't work. MN does live on as the progenitor of BitTorrent and Tahoe-LAFS, but the core idea of persistent, secure peer-to-peer storage across a dynamic peer group is a fools errand.

There are free systems that do this such as -

http://freenetproject.org/

http://oceanstore.cs.berkeley.edu/info/overview.html

Check out CrashPlan: http://www.crashplan.com

CrashPlan has a friend-net mode, but this is an entirely different kettle of fish than a truly dynamic peer network.

Reply on news.ycombinator.com