Hacker News Reader

what exactly is the objective here? just making sure there's multiple copies of the internet archive?

Nifty stuff in any case

There's multiple bits of objectives here - education, research, practice, and awareness. As time goes on, it's obvious that distributed sharing of data is the only real vaccine against the inherent problems of the modern Internet, and learning more about different ways this might be done benefit all.

Plus it's nice to have some of this data scattered about the world.

In the sphere of research & practice, are there any long-term plans for distributed processing applied to the data, akin to the Common Crawl in AWS allowing one to run map-reduce jobs against it?

There is a long history of sandboxing the JVM, which means in theory it should be safe to bring the code to the data without running the risk of having your local machine p0wned.

Reply on news.ycombinator.com