Don't know if this gets to the heart of your question, but I was surprised to not have seen https://www.scrapingbee.com/ mentioned here yet.