Hacker News Reader

Comment by Tuna-Fish | Hacker News Reader

The problem is not search, it's curation. Any system that accepts everything will be overrun with things you do not want (fakes, *very* illegal things), and so there needs to be some authority that determines what is allowed. At that point, you are centralized, and hosting a website doesn't significantly hurt you.

This is also the problem with all distributed social networks. In the end, your options are formal centralization, and informal centralization, because absolutely nobody wants to live in true decentralization.

View on HN

Has anyone tried to decentralize curation?

Here is straw man proposal, similar to cert chains and webs of trust: Say I'm a "curator". I say on HN/Reddit/Discord "here is my key hash 'p2pcuration:185da2bc59167692f596404fd83235f9bcb4e107b041f2e6e8d972da6dba00b7'". Any user that clicks the link or copies it into the search app adds the key to the trusted user list. With my private key I can sign torrents after I download them myself, which would mark the torrent as "good". When anyone who has added my key searches, the system searches for a corresponding signature from me as well. If a signature is found, the UI can chose to elevate that result.

The system could be extended so that signers could also sign other keys, expanding the trust network.

This system doesn't need to be run or maintained by each user. It could be served through a webui that can be run locally or shared with a small community. Migrating the interface to a new host would just require moving the config and keys.

What happens when you do this is that a few or one people in the system become trusted above the others, network effects do their thing, and the whole thing becomes informally centralized around them.

Most of the time when I search for something, I sort by more seeders first. Some type of signature can be used too.

> I sort by more seeders first.

But this couldn't be easier to game. All you have to do to be a seed is to report that you are a seed. You don't even have to send any data.

Sorting by number of seeders will probably get you a file that is fast to download because of the number of seeders, but it will rarely get you quality. Let's go take a stroll over to a popular public torrent site and search for, say, a recent superhero movie.

I see 100 (!) results for that movie's name. 49 of them have zero seeders at all. I don't know what even is the point. 29 of the results have one seeder. So already, 78% of the results are pure crap.

Let's look at the top result with 338 seeders: File is 3GB, H.264 video, 1080p, but with a crappy stereo AAC audio encoding... arrggh why??

Number 2 result with 84 seeders: 1.43GB, H.264 video, 720p, no word on the quality of the audio encoding. Even more worthless.

Number 3 result with 17 seeders: HEVC format, 2160p, audio streams include TrueHD Atmos 7.1, DTS-HD, Dolby Digital 5.1, stereo, and three non-English language streams. But, with an eye-watering download size of 61GB. Holy shit! Nice, but wow, what a download.

You have to go a few more down the list to find a good balance of high quality video and audio encoding, but with a reasonable file size. By that point you're in the single digit number of seeders.

Don't get me wrong, it's great to have a few choices and quality trade-offs. I guess there's someone out there who doesn't care about the stereo audio because they watch their movies with laptop speakers. But 100 results, with the vast majority of them either unseeded, poorly-seeded, or flawed in some way. I agree with OP: You definitely want some curation, not just search!

> arrggh why??

Because you are vastly overestimating how much most people actually notice or care about this.

I often pick the 0 seeders results if there is even a slight chance of better quality, and quite often there is.

0 seeders now doesn't mean forever. Some seeders only hook to the network once in a while, long enough for a few leechers to fetch it all. If the content is in demand enough then a few seeders may be left and suddenly a 0 seeder search result becomes an attractive one. Even curating is complicated for the human eye, so good luck automating that.

Yes it sometimes works but usually for smaller stuff like ebooks or music. You dont want to download gigabytes if not more from 1 spotty seeder, the chances it wont take days of 24h online and source being obline too are tiny.

Sometimes disconnect from site seeder count is bigger, and say 1 seeder mentioned is actually 15. But that was never rarbg's case, it was reliable and dependable like no other similar service. It also had IMBD rating so I could quickly weed out not so great stuff and focus on well rated ones, it generated maybe 60-80 movies a day with my filter applied, so quite a stream. After 2 week vacation, catching up took some time.

I used to, including today, to just go there every day or two and check whats new with my predefined filter. Often great movies that I never heard about before, old and new alike, took 2 mins to get 1080p x265 variant. Glad I've still managed to download that highly rated turkish movie this morning.

It is sad they didn't take the care to open source their stuff with whatever doc may be floating in their chats, for a resurrect

Great example. Some private trackers have strict rules about quality, e.g. at most one 720p and at most one 1080p (probably 1080p and 4k nowadays), bitrate must be above a certain threshold, whichever was published first stays, unless there is an "official" version published by the same people maintaining the website. The exact opposite of decentralized torrent sites, and works really well. Takes a lot of work, obviously.

I always assume that the movie studios themselves hired some proxies to put out those 61gb files so as to waste everyone's time and bandwidth.

This is a proven fact, do not be fooled. For decades Big Movie have been propagating disks out into the public with giant movie files on them.

Just like how Big Music decided to fight their pirates by only releasing unlistenable music for the last 20 years.

They are probably just ripped straight from a bluray without re-encoding.

> doesn't care about the stereo audio because they watch their movies with laptop speakers.

Stereo doesn't imply low fidelity. Just as 7.1 Dolby Unobtainium doesn't imply high fidelity speakers.

It's pretty much impossible to support Anonymous + Distributed + Free from poison/injection attacks. Not to mention, you still largely need at least some known points of entry into such a system.

> because absolutely nobody wants to live in true decentralization.

oh, some ppl do indeed want to.

thanks to encryption they can. but i agree that the likes of kazaa, edk and freenet are not for most ppl.

I seem to remember kazaa being extremely popular (especially after napster was gone) and moreover centralized. The client was local. gnutella sounds like what you're describing?

Some examples trying to deal with this involving Mastodon (years before the Twitter Musk debacle) :

Mastodon is big in Japan, and the reason why is uncomfortable ["child porn"]

https://news.ycombinator.com/item?id=15053064

Mastodon and the challenges of abuse in a federated system

https://news.ycombinator.com/item?id=17894684

How the biggest decentralized social network is dealing with its ["]Nazi["] problem

https://news.ycombinator.com/item?id=20429465

Trump’s new social media platform found using Mastodon code

https://news.ycombinator.com/item?id=29079404

Reply on news.ycombinator.com