ParentFull threadinovica·We use Scrapy for a few projects and it is really really good. They have a commercial-side to them, which is fine, but for anyone doing crawling/scraping I'd strongly recommend it. Good article alsoView on HN