Hacker News Reader

Ruby on Rails for the backend, Angular on the front. It's hosted on Heroku, it uses cloudfront, S3 for storage, Postgres for DB along with httpClient and mechanize for crawling and Paperclip for image manipulation.

View on HN

Do you use any kinda algorithm for crawling/fetching content from reddit?

Not particularly. The one thing that helps but most people don't know about reddit is the fact that adding a .json to the end of each url displays the content of that page in json format.

for example: reddit.com/r/funny.json

This make crawling/fetching content from reddit much more trivial than old school web crawling.

Reply on news.ycombinator.com