Ask HN: What are the most challenging pages to scrape?

2 pointsgbajson2 comments

Try PWA based one. Since they load in segments and cache a lot you'll have fun:

https://pwa.rocks/ - Look at the business and news webpages

Also look for the website that use AMP. They're even more fragmented then PWA pages. Below is article about AirBNB using AMP and iFrames:

I am looking for pages which are difficult to scrape, to see the applied techniques, and to learn how to bypass them.