Hacker News Reader

Comment by sounds | Hacker News Reader

Why is HN opposed to scraping? I think since the front page is dynamically generated -- a bot would waste resources. The site gets loaded (slow to respond) during peak times.

Beyond that, only pg could say...

View on HN

"... Why is HN opposed to scraping? ..."

because code written by N hackers scraping HN has a greater effect on the site, compared to using the reliable HNSearch API ~ http://www.hnsearch.com/api

  Ask HN: Does HN have an API and if not what's the etiquette for scraping?
  http://news.ycombinator.com/item?id=2138730

  Ask HN: Is there an API for HN?
  http://news.ycombinator.com/item?id=1107874

Judging from my own experience this API is not reliable.

Mine as well.

If you have problems with the API please let me know (andres@octopart.com).

Why would HN refuse to provide a useful RSS feed / a non standard compliant one (number of comments/points/time of submission/id) that could be used to develop apps for a better reading experience? http://api.ihackernews.com/ is unreliable at best.

I'm working on something right now and this limitation causes difficulties. (http://cl.ly/241x3F3d123P0w0L1s3q)

I do HN scrapping of my 'saved stories' for my personal site http://zalew.net/hn/ so I assume pg doesn't mind that as it scraps only once per hour. here's the description of it and source http://zalew.net/2011/12/08/grab-your-hackernews-stories-and...

Reply on news.ycombinator.com