What I need is an API scraper, scrapy seems to be mostly for HTML. I know how to look at network requests in Chrome dev tools and JS function to understand the shape of REST API, so I need something to plan the exploration of the arguments space. For example if you want to scrap airbnb, you look at their API, find there is a REST call with a lat, long box, I need something to automatically explore an area, if the api only give the 50 first results and you hit this number of calls it should schedule 4 calls with half the size boxes and so on. If the request has cursors, you should be able to indicate to the scraper how to follow it. I don't know what is the best tool for that.