HNHacker News
TopNewBestAskShowJobs

liberalgeneral

222 karma · joined August 21, 2022

submissionscomments
liberalgeneral··on LibGen's Bloat Problem
My target audience is not normal people though, and I don't mean this in the "edgy" sense. The fact that we are having this discussion is very abnormal to begin with, and I think it's great that there are some deviants from the norm who care about the longevity of such projects.

I can imagine many students and researchers hosting a mirror of LibGen for their fellows for example.

liberalgeneral··on LibGen's Bloat Problem
Z-Library has been innovating a great deal in that regard. Sadly they are not as open/sharing as LibGen mirrors in giving back to the community (in terms of database dumps, torrents, and source code).
liberalgeneral··on LibGen's Bloat Problem
I think Sci-Hub is the opposite since 1 DOI = 1 PDF in its canonical form (straight from the publisher) so neither duplication nor low-quality is the case.
liberalgeneral··on LibGen's Bloat Problem
You are right, added a paragraph at the end.
liberalgeneral··on LibGen's Bloat Problem
I don't think you've deserved the downvotes, and I don't think it's a bad idea either; indeed some coordination as to how to seed the collection is really needed.

For instance phillm.net maintains a dynamically updated list of LibGen and Sci-Hub torrents with less than 3 seeders so that people can pick some at random and start seeding: https://phillm.net/libgen-seeds-needed.php

liberalgeneral··on LibGen's Bloat Problem
Database dumps are available here if you are interested: http://libgen.rs/dbdumps/

libgen_compact_* is what you are probably looking for, but they are all SQL dumps so you'll need to import them into MySQL first. :/

liberalgeneral··on LibGen's Bloat Problem
https://news.ycombinator.com/item?id=32540854
liberalgeneral··on LibGen's Bloat Problem
> While I'd love to mirror whole archive locally, it would really be superfluous because I can only read a couple of quality books at a time anyway, [...]

I'd love to agree but as a matter of fact LibGen and Sci-Hub are (forced to be) "pirates" and they are more vulnerable to takedowns than other websites. So while I feel no need to maintain a local copy of Wikipedia, since I'm relatively certain that it'll be alive in the next decade, I cannot say the same about those two with the same certainty (not that I think there are any imminent threats to either, just reasoning a priori).

liberalgeneral··on LibGen's Bloat Problem
> Has anyone ever stumbled across an executable on LibGen? The article mentioned finding them but I've never seen one.

Here is a list of .exe files in LibGen: https://paste.debian.net/hidden/1c82739a/

And a breakdown of file extensions: https://paste.debian.net/hidden/579e319c/

> And would be nice to have a local copy of most of the books.

Yes! That was my intention—I wasn't advocating for a purge of content but a leaner and more practical version would be amazing.

liberalgeneral··on LibGen's Bloat Problem
Thank you for your efforts!

To be clear, I am not advocating for the removal of any files larger than 30 MiB (or any other arbitrary hard limits). It'd be great of course to flag large files for further review, but the current software doesn't do a great job at crowdsourcing these kinds of tasks (another one being deduplication) sadly.

Given the very little amount of volunteer-power, I'm suggesting that a "lean edition" of LibGen can still be immensely useful to many people.

liberalgeneral··on LibGen's Bloat Problem
If you are referring to my duplication comments, sure (but even then I believe there are duplicates of the exact same edition of the same book). Though the filtering by filesize is orthogonal to editions etc. so has nothing to do with that.
liberalgeneral··on LibGen's Bloat Problem
Here is the raw data if you are interested: https://paste.debian.net/hidden/77876d00/