Hacker News Reader

Comment by wesleytodd | Hacker News Reader

I think the faster we make companies liable for the content on their servers the better. Why? Because I want distributed/federated data to become an actual priority to these companies. I want it to hit their bottom line when they get sued for content uploaded to their servers so they consider keeping the data owned by the user who generated it.

View on HN

Surely you realize that making data a liability reduces the ability for you and I to host it, not increases it. I understand you are against centralization of data, but you can't magically commoditize and distribute it via punishments. You'll only strengthen those that can take a hit. While it seems mathematically logical to say "well, if we make it less financially beneficial to have data, they won't have it" what happens in practice is "well, if we make it less financially beneficial to have data, nobody will have it". There are alternatives to decentralization that have less effect on smaller orgs.

If anything, I think liability would be a forcing function that would strongly encourage centralization. Only those with deep pockets could afford it.

Yeah I don't want to be hosting data I have not vetted either. I think being aware of the content any device you own is serving on the net is a good thing, even if it reduces the amount of people willing to do it for others.

TBQH if small companies cannot host user data that is good too. It should be very difficult for companies of any size to house data they are not willing to defend legally.

Also, I am quite aware of how disruptive this ideal would be to most of our current large internet companies. I don't think it will realistically change any time soon, and am sure it would have side effects which would have to be balanced. But the direction we are headed now seems much worse to me than the alternatives.

> TBQH if small companies cannot host user data that is good too. It should be very difficult for companies of any size to house data they are not willing to defend legally.

We just disagree here. We know who can and who can't defend things legally. On a personal level, I don't like my options further reduced for who I can willingly give my data to. Legislate and determine what constitutes private data, data misuse, and/or surreptitious gathering if you want, but non-private information freely and consentually given should want to be freely proliferated.

My argument is more that forcing tech companies to invest in distributed data ownership tech would push that forward in a way that would enable more competition of services because your data would be portable, not locked into vendors like it is now.

In the short term it might result in less variety, but only until the tech proliferated and reached maturity. Then services could simply spin up on top of the data that you already host yourself. So you would have to take on the responsibility of finding a good way to host the data you want available (probably from your phone), but after that you would be able to use any service willing to operate on your data, but your data would be in public standards, not proprietary storage inside a facebook/google/amazon data center.

I’ve been working on that. It’s a hard problem, but emule works quite well. I propose something based on that: when you start a program, you advertise all the sha256’s of all the files you share. Whenever you need a resource, you send a request to the dht for its sha256. Anyone who has the file will opportunistically try to send it to you.

Since it’s a dht, there’s nothing to take down. And since it operates on sha256s of data, it’s implicitly secure. And it uses tor for the rendezvous, so it’s not possible to track who is requesting what, except by unique ID (which can just be a bitcoin wallet address you control).

The hard part is, what do you do about abuse? What if someone spams the network with bots that try to fulfill every request with bad data?

If anyone has research refs in this direction, I’d be grateful to read them. I’d also like to avoid a blockchain if possible, since it seems unnecessary for simple federated distributed data.

Check out Dat: https://datproject.org/

Excellent: https://raw.githubusercontent.com/datprotocol/whitepaper/mas...

It seemed too simple of an idea not to already be done. Hopefully it conceals your IP.

Arg. Nope. It does not. It leaves it up to the client to decide how best to request the content, and by default that almost certainly guarantees no privacy.

It was not intended to provide privacy, the the distributed data store. If you want privacy you probably just want to have a private swarm, which is perfectly fine to setup. Just implement a client with a whitelist or another mechanism for the privacy you want.

If there is no privacy, why not host the data on s3 or Dropbox? Data storage is dirt cheap. The privacy concern seems like the prime reason to build such a service.

Here is an interesting take on this "why?" by @pfrazee:

https://www.youtube.com/watch?list=PL37ZVnwpeshG2YXJkun_lyNT...

I think a cool layer on top of this would be to create a P2P network where you can 'friend' certain peers and produce a reddit-like aggregation of new content from various sources.

- Friend another user by adding their key and IP to a trust group.

- When your client boots up, it connects to friend nodes and downloads their recent DHT.

- DHTs from all friends are summed to sort the hashes by descending frequency, perhaps with a time decay factor.

- Display reddit-like UI of content to User: see which files are recently most popular among the set of all your friends.

Possible improvements:

- User could customize time vs vote weighting, script custom sort rules

- Give certain peers greater vote weighting ('best friend' vs 'acquaintance')

- Allow peers to 'tag' hashes to create subreddit like collections of things.

- Allow attachment of metadata to hashes - title, description, etc - figuring out how to handle discrepancies between sources here could be tricky.

- Make client automatically re-host content in your DHT: hosting IS the upvote

Check out beaker browser and fritter:

https://beakerbrowser.com/ https://github.com/beakerbrowser/fritter

Built on top of dat, it is basically this kind of architecture.

Cool, I'll have to look into this more, thanks.

You and I must have like minds. I have even created an anon DHT PoC [0], conceptualized how the messaging might work [1], made a lib to use Tor easier since it's the best anon nat buster these days [2], began toying with a superset of reddit/slack/forums/etc (some grpc files at [3] and some impl in that same repo), and a bunch of other small things in order to arrive at this final destination.

0 - https://github.com/cretz/tor-dht-poc 1 - https://github.com/cretz/software-ideas/issues/6 2 - https://github.com/cretz/bine 3 - https://github.com/cretz/yukup/tree/master/yukup/pb/proto

Yesss, [0] saves me so much work if it does what the label says. Thanks!

Very cool! I'll have to dig into this a bit more thanks!

Check out ethereum swarm [1]. Optionally private (hosting and accessing), decentralized, p2p data hosting protocol with baked in payments if you want to pay the network to host your data for you.

[1] https://swarm-guide.readthedocs.io/en/latest/introduction.ht...

Reply on news.ycombinator.com