Hacker News Reader

Comment by kazinator | Hacker News Reader

Last year, I read your post and implemented it. I had no way to tell whether there is any benefit to it, so I discontinued it. Serving the data, even though it is compressed and the compression is precomputed, was still a waste of bandwidth. It could hit the browsers of legit user (false positive).

The main problem on my site is that bots hit the CGIT repo, where visiting files triggers syntax highlighting scripts.

I put in a skill-testing-question type system where you get a maximally long-lived access cookie if you answer right (so legit users are bothered as infrequently as possible).

View on HN

Have you thought about adding a proof of work challenge? Your bandwidth is spared and attackers get increasingly expensive computational puzzles.

Disclosure: I work for Friendly Captcha, which uses the above strategy.

Your syntax highlighting is neither client-side nor pre-rendered?

CGIT doesn't support client-side syntax highlighting, as far as I know. Anything could be hacked, but out of the box, it has a system consisting of dispatching a script.

I have custom syntax highlighting. For instance, I have a home-grown script which takes a man page file (identified by a suffix like .1) and renders it readably, so it looks like a man page on your terminal, rather than nroff source code. (The only unsolved problem is that CGIT renders a column of the original line numbers down the left, which exceeds the length of the rendered page.) I also have accurate syntax highlighting for a home-grown programming language. That syntax highlighting exists only for Vim, which has a superb syntax highlighting engine. So in fact Vim is launched, via expect, to do some HTML rendering! There is no way that is getting translated into client-side.

Pre-rendering is not really an option over numerous git repos, that have numerous files in them, with the added dimension of history. What CGIT has instead is tunable caching.

Bots which aggressively scrape the history are hostile to the caching scheme, however. It works best for human visitors which mostly look at the head of the master branch in each repo, and poke around just a little bit in the history or look at other branches.

Reply on news.ycombinator.com