You sound like Elon. LIDAR is far superior to radar in a lot of cases. The best situation would be for Tesla's to use a mix of both radar and lidar.
2 karma · joined July 29, 2019
What's the plan here? Check for a sitemap.xml (which generally only contains crawlable URLs anyway) or crawl the index and look for all links and send a request to your service for every URL before crawling it?
I personally think it would be better suited as a library where you can pass it a robots.txt and it'll let you know if you can crawl a URL based on that.