Hacker News Reader

Comment by moebrowne | Hacker News Reader

This kind of thing can be mitigated by not publishing a page/download for every single branch, commit and diff in a repo.

Make only the HEAD of each branch available. Anyone who wants more detail has to clone it and view it with their favourite git client.

For example https://mitxela.com/projects/web-git-sum (https://git.mitxela.com/)

View on HN

Alternatively, from the nginx config file for git.ardour.org:

   location ~ commit/* {
        return 404;
    }

I got another interesting idea from this and another comment but what if we combine this with ssh git clients/websites with the normal ability.

maybe something like https://ssheasy.com/ or similar could also be used? or maybe even a gotty/xterm instance which could automatically ssh/get a tui like interface.

I feel as if this would for all scrapers be enough?

i'm working on something similar: instead of web-based ssh client, it's a web-based git client UI - you can "checkout" repos, browse commits, tree, read individual files, etc. with no server-side code at all; git objects are fetched and parsed on client-side. first target is the dumb-http git protocol, so people can host git repos on static websites, and visitors don´t need to clone by a local git client to peek in.

https://bandie91.github.io/dumb-http-git-browser-js-app/ui.h...

I suspect bog-standard per-IP rate limiting would also mitigate this, no?

People report that the requests come from many thousands of IPs

Any firewall worth its salt (like PF) can measure which IPs are making more requests per minute than is reasonable and shove them into a list to be handled separately (whether by blocking them outright or by putting them in a slower / lower-priority queue). Putting all those thousands of IPs into a queue that only gets 1% of the available bandwidth would solve the issue quite nicely, I think.

Reply on news.ycombinator.com