Full threadtiborsaas·I've just discovered Headless Chrome crawler and it works pretty well. Not sure how well it will scale, but I'll index a few hundred sites only.https://github.com/yujiosaka/headless-chrome-crawlerView on HN