Show HN: Torrentinim – low memory-footprint, API-only torrent search engine | Hacker News Reader

Show HN: Torrentinim – low memory-footprint, API-only torrent search engine

github.com

117 pointssergiotapia31 comments

This queries the tracker websites, but I was hoping this would be a distributed hash table (DHT) search engine, like bitcq. Bitcq is indispensible for obscure torrents, but its interface could be better.

Edit: wait, looking at the source, this doesn't search torrents on the mentioned trackers, it keeps an eye on feeds of newest or most popular (in case of tpb) torrents and caches the tracker metadata. This doesn't seem very useful to me. Over a long time you'll have a big view of the tracker's database, but not older content than you started tracking, and, in the case of tpb, you'll just have a view of torrents that have been in the top 100 since you started crawling. Weird. Also, peer counts don't seem to get updated, I think.

I wanted a very small, efficient program that anyone could run without ceremony on their machine. Nim made it very easy for me to do. It looks like Python/Ruby, runs really fast, and uses very little resources. I love Nim!

> It looks like Python/Ruby, runs really fast, and uses very little resources.

You should also check out Crystal, which had its 1.0 release earlier this year: https://crystal-lang.org/ (Although, no Windows support yet.)

Impressive!

> High performance, extremely low memory footprint. > RAM usage (avg.): 21.5MB > Binary application size: 600KB

Application size here[1] is 1.8MiB, Ram 131MiB (release build -> make build)

[1] Freebsd13 x64, Nim Compiler Version 1.4.8

This looks nice! Is there a way to configure it to exclude certain categories of content (such as nsfw)? Do you tend to run this behind a proxy?

After a brief look at the code it doesn't seem so, however there's a procedure (insert_torrent() in src/torrents.nim) in which a retrieved torrent is added to the database, of which one column is the name of the torrent, so it would be possible to insert a filter based on a list of nsfw words that would discard the torrent if one or more are found. Of course it wouldn't work on torrents that hide their nature, which hopefully happens very rarely.

Does anyone have a book recommendation to learn and implement the bittorrent protocol myself ? Ive been looking for a good book on the protocol.

Not a book but I wrote an article about implementing the basics of a BitTorrent client in Python a few years ago. Maybe you'll find that useful. https://markuseliasson.se/article/bittorrent-in-python/

Thanks a lot ! Really appreciate it

Interesting project, what are some similar self-hosted solutions that allow you to search torrents on public trackers?

To the author: How do you personally use it, do you have a custom interface to search the aggregated results or do you just plug it into a bigger content consumption system?

> what are some similar self-hosted solutions that allow you to search torrents on public trackers?

Jackett. https://github.com/Jackett/Jackett

also Prowlarr https://github.com/Prowlarr/Prowlarr

also Magnetissimo https://github.com/sergiotapia/magnetissimo

implemented using Elixir and Phoenix. Cool little project

It's a former project by OP, who stopped work on it: > I've paused development of Magnetissimo to focus on Torrentinim.

Good catch, didn't notice that!

Magnetico[1] is an easy-to-use, self-hosted DHT crawler. It has a web UI that shows statistics and gives you a search engine (it uses sqlite full-text search).

It needs quite a bit of bandwidth and some storage space, but worked well so far, for me. I've been running it for a couple of years and it indexed around 12.5M torrents (36GB of uncompressed database).

[1]: https://github.com/boramalper/magnetico

Do you mind sharing what sample daily/monthly bandwidth during discovery looks like?

You can somewhat control the bandwith by limiting the number of simultaneous peers: in my case the usage seems to be around 5 Mbps up/down.

If you're running it from home, the real problem is not the bandwidth but the number of open connections the router has to keep track (assuming the network is NATed). The default settings will likely crash any cheap consumer router, so I would run the daemon from a VPS or a machine that isn't behind a NAT.

I use it to search for DODI releases and x265 releases, it's faster than looking through websites. Once I finish Radarr/Sonnar support, I'll point my instances at Torrentinim.

I liked magnetissimo (from the same author) much more, but its kind of sleeping:

https://github.com/sergiotapia/magnetissimo

As an aside, does anyone know of a light-weight torrent tracker that can be used for personal use?

I always wanted to setup a tracker for documents and files etc, without the need for huge member management systems etc.

If you want a hosted tracker instead, I’ve used privtracker.com for personal files with success. If you want to set one up, chihaya or udpt might do the job.

what about Resilio Sync?

^ Or it's open source version, SyncThing.

OpenTracker and XBT are the most famous for lightweight use.

OpenTracker is absolutely great and under one of the best licenses -> Beerware

https://erdgeist.org/arts/software/opentracker/

Hard to get simpler than BitStorm. A single PHP file you can drop on a web server: https://web.archive.org/web/20111101104813id_/https://ck3r.o...

I've been looking for something like this but on the JVM (specifically Java or Scala if possible)

What do people use this for? I mean beside creating another pirate bay (CIMMW, that's the only thing comes to mind when I see "torrent search engine").

Dumb question I'm sure, but what does CIMMW stand for? (search results didn't turn anything up for me)

Reply on news.ycombinator.com