Hacker News Reader

A couple months ago, reddit started refusing requests from my web scraper. Figured out they started checking the user agent and refusing connections that didn't look like they came from a user's browser. Unless I missed an announcement somewhere, it doesn't seem like they're overly friendly about allowing web scrapers.

View on HN

They just don't want to be abused.

"We're happy to have API clients, crawlers, scrapers, and Greasemonkey scripts, but they have to obey some rules:"

https://github.com/reddit/reddit/wiki/API

They are fine with it as long as you abide to their terms, they have a subreddit dedicated to reddit development and the reddit api which has discussion of scraping: http://www.reddit.com/r/redditdev

Reply on news.ycombinator.com