Crawling the World Wild Web

In this post we discuss some of the existing technologies for scraping and parsing web pages. We also talk about some of the challenges developers may encounter while scraping dynamic web pages.