The main issue in scraping:
- If you scrape a lot, you will be block based on you IP; You need to use PROXY - Scraping entire website need specific logic, retries and more - It becomes an heavy background job
All the above takes time, so if in your business it is not your core feature, likely better to outsource it.
Good job doing it tho!