Robots.txt – Just Crawl It | Hacker News Reader